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PREFACE 

This  Memorandum  Is  a  by-product  of  an  investigation  of  human  factor 
aspects  of  automatic  checkout  equipment  done  by  Dunlap  and  Associates, 

Inc.,  for  RAND's  Project  ACE.  The  bibliographical  material  appeared  to 
be  useful  in  its  own  right;  hence  this  Memorandum,  which  retains  the 
informal  style  of  the  original  compilation. 

The  material  should  be  helpful  to  system  designers  and  military 
purchasers  and  users  of  automatic  checkout  equipment. 

* 

Distribution  follows  the  general  pattern  set  by  other  ACE  publications, 
and  is  directed  primarily  at  Air  Force  and  other  military  agencies  and 
industrial  contractors  concerned  with  automatic  checkout  equipment. 


RM-2750,  An  Omnibus  of  Briefing  Papers  on  Analysis  of  Automatic 
Checkout  Equipment  and  Aids  to  Its  Design,  suamarizee  much  of  the  Project  •  s 
results,  am  Incorporates  a  list  of'  tije  Project  ACE  publications. 
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I .  INTRODUCTION 

This  annotated  bibliography  was  prepared  by  Dunlap  and  Associates,  Inc., 
under  contract  to  The  RAND  Corporation.  The  objective  of  the  contract  was 
to  investigate  human  factors  aspects  of  automatic  checkout  equipment  (ACE); 
major  results  are  presented  in  a  separate  report  (Item  24,  p.  22  ).  During 
the  study,  documents  relating  to  ACE,  maintainability,  and  related  subjects 
were  collected  and  abstracted.  The  compilers  selected  for  this  work  those 
publications  they  deemed  most  pertinent  to  the  objectives  of  Project  ACE. 

Though  the  literature  can  be  classified  in  many  different  ways  and 
some  overlap  among  classes  is  inevitable,  it  was  found  that  a  simple  five- 
category  system  was  useful  in  organizing  the  documents .  Die  five  subject 
areas  are: 

1.  General  philosophy  and  review  of  automatic  checkout  methods 

2.  Maintainability  directives  and  guides 

3.  Models  of  the  maintenance  process 

4.  Human  performance  of  checkout  and  fault-isolation  tasks 

5.  Specific  checkout  systems 

The  general  articles  on  checkout  philosophy  have  much  in  comaon.  They 
usually  assert  that  checkout  considerations  must  be  worked  in  carefully  from 
the  earliest  design  phases,  that  design  for  maintainability  is  a  good  thing, 
that  some  ACE  installations  have  been  disappointing,  and  that  no  across- 
the-board  or  universal  checkout  arrangement  can  fit  every  situation.  Some 
papers  delineate  the  significant  variables  in  setting  up  a  checkout  concept 
(e.g.,  Items  10  and  13 )  and  some  predict  the  future  of  ACE. 

Maintainability  guides  have  been  written  for  most  of  the  R&D  pro¬ 
curement  agencies  in  the  Department  of  Defense,  and  the  tendency  is  toward 
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including  a  strong  maintainability  directive  in  new  equipment  contracts 
(see  Item  28).  Several  big  weapon  systems  have  their  own  maintainability 
manuals.  Typically,  a  guide  consists  of  an  annotated  checklist  of  design 
considerations,  with  sections  on  test  equipment,  tools,  manuals,  labeling, 
access,  work,  environment,  and  procedural  factors.  To  make  an  equipment 
maintainable,  one  works  through  the  listed  details  and  corrects  unsatisfactory 
features  until  guide  reconmendations  have  been  met. 

Models  of  the  maintenance  process  attempt  to  formalize  relations  among 
a  few  supposedly  central  variables  and  to  operate  on  these  relations.  Impli¬ 
cations  of  the  model  can  then  be  derived,  different  sets  of  assumptions  or 
constraints  evaluated,  and  so  forth.  Most  models  lean  toward  time  definitions 
of  model  parameters  and  are  concerned  with  the  dependencies  among  reliability, 
maintainability,  and  availability . 

The  typical  paper  on  human  performance  of  maintenance  tasks  reports 
time  or  other  scores  for  military  technicians  who  were  required  to  locate 
selected  malfunctions.  Some  papers  are  also  concerned  with  ancillary  issues, 
such  as  the  improvement  of  maintenance  ability  by  aids  or  training,  factors 
that  make  troubles  easy  or  difficult  to  solve,  and  means  by  which  individual 
performance  can  be  predicted. 

Manuals  on  specific  ACE  configurations  are  generally  descriptive  and 
straightforward.  Very  few  of  the  ACE  manuals  clearly  relate  checkout  re¬ 
quirements,  indicating  that  the  ACE  design  group  became  active  only  after 
weapon - sy st em  and  maintenance  concepts  were  already  settled. 

The  abstracts  vary  in  length  from  one  sentence  to  several  pages. 

Length  per  se  has  nothing  to  do  with  the  quality  or  significance  of  an  item; 
the  abstracts  were  written  to  facilitate  preparation  of  the  main  report  and 
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in  some  cases,  a  detailed  vrite-up  seemed  more  useful  than  a  brief  summary 
statement. 

The  scope  of  the  project  did  not  permit  an  exhaustive  bibliographic 
effort.  The  abstractors  continued  until  they  believed  they  had  examined 
most  of  the  major  documents,  and  began  to  encounter  obvious  redundancy 
and  sharply  diminishing  returns .  References  found  after  the  bulk  of  the 
abstracting  had  been  completed  are  listed  at  the  end  of  Secs.  Ill,  IV,  V, 
and  VI. 
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1.  Aldrich,  J.  A.,  "Automatic  Checkout  Equipment:  An  Airline  Viewpoint," 
paper  read  at  SAE  National  Aeronautic  Meeting,  Los  Angeles,  October  10-14, 
I960. 

Airline  servicing  of  flight  equipment  is  split  into  two  main  categories: 
(l)  line  or  day-to-day  maintenance,  and  (2)  periodic  overhaul.  Because 
of  the  tremendous  cost  of  the  aircraft,  down  hours  are  very  expensive 
and  for  this  reason  ACE  has  been  considered  seriously  as  a  means  of 
improving  maintenance  efficiency. 

Despite  this  possibility,  ACE  manufacturers  have  not  conmamicated  fully 
with  the  airlines.  The  airlines'  management  ie  not  yet  convinced  that 
military  types  of  checkout  equipment  are  appropriate.  In  many  cases  the 
original  airborne  equipment  has  not  been  designed  with  ACE  in  mind,  so 
that  extensive  rewiring  would  be  necessary  before  ACE  could  be  installed. 

Two  recent  checkout  projects  for  the  DC-8  transport  are  described.  One 
is  a  semi-automatic  "suitcase  tester,"  designed  to  perform  99  distinct 
tests,  for  the  automatic  pilot.  Another  tester  is  being  built  for  the 
main  a.c.  generating  system.  Full  utilization  has  not  been  achieved  in 
either  case. 

The  author  believes  ACE  should  be  provided  for  the  complex  installed 
systems  only:  "It  would  be  foolish  and  wasteful  to  try  to  employ  such 
elaborate  equipment  for  any  and  every  unit  and  system  on  the  airplane 
....We  have  discouraged  the  concept  of  a  large  van  or  truck  full  of 
electronic  equipment  drawing  up  alongside  the  monster  Jet  aircraft  and 
plugging  in  a  tremendous  cable  to  a  receptacle  on  the  side  of  the  airplane 
to  check  out  the  entire  vehicle." 

ACE  can  be  an  effective  add  when  new  aircraft  models  are  being  phased  in; 
as  the  new  systems  become  familiar  to  the  maintenance  crews,  short  cuts 
can  be  taken  and  "it  would  be  natural  to  assume  that  ACE  would  be  used 
somewhat  less  frequently." 

The  "black  box"  system  of  module  replacement  is  far  too  costly  for  the 
airlines.  Therefore,  ACE  that  could  troubleshoot  down  to  the  submodule 
would  be  very  useful.  Equipment  that  could  test  several  details  on  a 
single  system  is  px’eferable  to  equipment  that  can  accomplish  gross  checks 
on  a  number  of  systems.  Equipment  must  be  convenient  to  connect,  easy 
to  Interpret,  and  manageable  in  terms  of  weight  and  bulk. 


2.  Argonne  National  Laboratory,  Proceedings  of  a  Symposium  on  Large  Scale 
Digital  Computing  Machinery,  November,  1953>  295  PP- 

One  item  of  this  collection  is  relevant  to  checkout.  Goldstone's  paper, 
entitled  "Some  Experiences  in  Coding  and  Programming  with  the  Institute 
Computer,"  describes  some  techniques  for  rapid  checking  of  programs. 
Presumably  these  techniques  would  be  applicable  to  self -check  and  fault 
locator  routines. 
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3.  Bock,  L.  L.,  and  M.  Macauley,  Computer  System  Reliability,  Engineering 
Research  Associates,  1953- 

This  early  paper  treats  the  problems  that  arise  in  achieving  reliable 
digital  computer  operation.  The  techniques  adopted  for  Uni vac  I  are 
described.  Reliability  being  an  unknown,  the  equipment  was  cautiously 
designed  to  include  as  many  checks  and  redundancies  as  possible.  This 
design  philosophy  yielded  a  high  percentage  of  operating  time  compared 
to  down  time.  The  paper  gives  some  statistics  on  operating  time  and 
reviews  the  design  principles  involved  in  circuit  redundancy. 


4.  Briggs,  S.  J.,  Presentation  to  Second  Human  Engineering  Work  Group 

Meeting  FEK  System,  Nortronics  NSS  Report  No.  7^9,  October,  I960,  8  pp. 

Cost  figures  for  maintainability  are  presented.  "Air  Force  studies  show 
that  the  direct  cost  of  maintaining  equipment  runs  about  two  dollars  per 
year  for  every  dollar's  worth  of  equipment.  This  is  only  direct  cost, 
that  is,  the  cost  of  operating  the  various  depots,  field  shops  and  flight 
line  activities,  and  the  cost  of  replacement  parts.  This  does  not  in¬ 
clude  the  indirect  costs  of  training,  logistics,  and  other  indirect 
costs.... In  reality, .. .the  true  cost  of  maintaining  a  dollar's  worth 
of  military  electronic  equipment  is  probably  closer  to  $20  than  it  is  to 
$2. 

"From  the  USAF  Electronics  Maintenance  Conference  we  learn  that  at  present, 
one  out  of  eight  interceptor  aircraft  is  available  for  use.  The  other 
seven  are  'being  maintained' ...  .For  the  B-52  system, ..  .about  37-5  hours 
flight  line  maintenance  time  is  spent  per  flight  hour." 

Three  major  maintainability  requirements  are  logistic,  training,  and 
time.  "Logistic  requirements  are  reduced  as  component  repair  work  is 
done  closer  to  the  operating  equipment."  Time  and  training  requirements 
are  closely  related  because  "any  systematic  approach  to  time  requirement 
reduction  must  include  an  analysis  of  maintenance  procedures  with  simpli¬ 
fication  as  a  goal.  Such  an  analysis  will  result  in  the  elimination  of 
irrelevant  Job  activities,  the  simplification  of  decisions,  and  the 
inclusion  of  well  designed  Job  aids.  All  of  these  tend  to  reduce  training 
requirements." 

Nine  kinds  of  down  time  are  listed: 

1.  Fault  detection  time 

2.  Waiting  time  for  technician 

3.  Fault  verification  time 

4.  Access  and  preparation  time 

5.  Fault  localization  time 

6.  Waiting  time  for  spare  parts 

7.  Fault  correction  time 

8.  Alignment  and  adjustment  time 

9.  Checkout  time 
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"Between  60  and  80  per  cent  of  the  (total)  time  is  spent  on  two  of  the 
listed  items,  namely,  fault  location  and  access."  Ways  of  reducing 
those  times  are  illustrated.  The  access  problem  is  the  simpler  of  the 
two  and  can  be  handled  by  improved  standard  connectors  and  fasteners. 

In  the  fault  isolation  problem,  techniques  such  as  self -test,  error 
detection,  and  module  output  quality  checks  are  useful  in  streamlining 
the  work.  It  is  reconmended  that  a  definite  fault -to - re suaed-operat i on 
time  requirement  be  imposed  on  the  system  designer;  a  15-minute  goal 
was  set  up  for  the  Polaris  missile.  Similar  standards  for  "component- 
fix-time"  are  also  expected  to  be  useful. 


5.  Bright,  J.  R.,  Automation  and  Management,  Harvard  University  Press, 

1958,  270  pp. 

In  1954,  a  project  to  study  the  impact  of  automatic  data  processing  was 
begun  at  the  Harvard  Graduate  School  of  Business  Administration.  This 
volume  reports  detailed  results  from  thirteen  automation  programs  which 
were  carried  out  in  American  industry.  Seventeen  levels  of  mechani¬ 
zation  are  defined  in  such  a  way  that  "mechanization  profiles"  can  be 
drawn  for  a  manufacturing  sequence.  Among  the  many  automation  issues 
treated  are  expected-versus-reallzed  advantages,  installation  and 
procurement  problems,  the  impact  on  sales  and  work  force,  and  organizing 
for  automatic  production. 


6.  Bull,  W.  I.,  "The  Navy  Outlook  on  Maintainability,"  in  F.  L.  Ankenbrandt 
(ed. ),  Electronic  Maintainability,  Vol.  3*  Reinhold,  New  York,  i960. 

"Any  discussion  of  maintenance  must  concern  itself  with  the  four  funda¬ 
mental  elements  involved:  the  man,  the  maintenance  manual,  the  test 
equipment,  and  the  repair  parts."  The  P0M3EE  maintenance  standard  books 
are  described;  a  special  book  for  each  piece  of  equipment  shows  the 
technician  how  to  accomplish  the  required  checks.  Spaces  are  provided 
in  the  book  for  entering  parametric  values,  so  that  a  permanent  per¬ 
formance  record  can  be  made.  Standard  measurements  for  that  particular 
cqulprvfnt  item  are  entered  at  the  shipyard  when  the  equipment  is  assumed 
to  be  in  peak  condition. 

Airborne  and  missile  equipment  tend  to  have  replaceable  (but  repairable) 
modules  and  automatic  checkout  tests.  "For  all  types  of  equipment  in 
the  Navy,  it  is  quite  obvious  that  there  will  be  a  marked  Increase  in 
the  use  of  built-in  test  features  and  automatic  fault  locators.  Fbr 
extremely  complex  equipments  this  is  fast  becoming  essential.  And  of 
course,  the  need  is  compounded  by  the  shortage  of  competent  military 
technicians."  However,  "we  will  still  have  to  concern  ourselves  with 
maintenance  policies  and  depend  heavily  on  the  technician  and  his  test 
equipment  to  keep  our  complex  systems  operable." 


7.  Cooper,  J.  I.,  L.  V.  Rigby,  and  W.  A.  Spickard,  Proceedings  of  Shirt 
Sleeve  Seminar  on  Maintainability,  Northrop  Corporation,  Noredr  Report 
NOR-&) -3201,  August,  i960,  118  pp. 
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This  document  is  a  verbatim  record  of  a  seminar  in  which  twenty-four 
human  factors  and  design  engineering  specialists  discussed  maintain¬ 
ability  problems.  The  following  paragraphs  indicate  some  of  the  central 
themes  relating  to  test  and  checkout  operations.  The  quotes  should  not 
be  construed  as  reflecting  the  unanimous  opinion  of  the  participants. 

There  are  four  major  maintainability  problems:  (l)  how  to  design  for 
maintainability;  (2)  how  to  determine  the  degree  of  maintainability 
that  has  been  provided;  (3)  how  to  choose  the  right  desigp  among  several 
alternatives;  and  (4)  who  should  implement  and  be  responsible  for 
maintainability  design.  There  are  now  several  guides  for  the  first. 

The  second  is  still  very  much  a  problem.  The  third  is  an  operations 
analysis  problem  that  needs  inputs  from  those  who  are  in  a  position  to 
estimate  trade-offs  between  maintainability  design  features  and  other 
weapon  system  goals.  The  fourth  seems  to  require  a  special  maintain¬ 
ability  group. 

Reliability  and  maintainability  are  not  independent.  Equipment  that 
is  almost  100  per  cent  reliable  still  requires  preventive  maintenance, 
or  its  reliability  will  be  degraded.  Thus,  even  in  highly  reliable 
weapon  systems,  there  will  still  be  a  need  for  good  maintainability 
design. 

The  Systems  Analysis  and  Integration  Model  (SAIM)  (see  Item  51)  can  be 
used  to  show  the  interactions  between  maintainability  and  other  system 
elements.  SAIM  is  a  general  system  model  in  which  the  system  is  composed 
of  (1)  external  determinants,  which  constrain  the  Bystem;  (2)  components, 
which  are  the  men,  machine,  and  facilities;  and  (3)  integrative  elements, 
which  bind  the  system  components  together.  Another  matrix,  similar  in 
appearance  to  SAIM,  can  be  used  to  index  detailed  maintainability  design 
information.  This  matrix  has  three  dimensions:  (l)  equipment  attributes 
(coding,  connectors,  controls  displays,  external  accessibility,  fasteners, 
etc.);  (2)  maintenance  actions  (malfunction  recognition,  diagnosis, 
servicing,  adjusting,  removing,  replacing,  repairing,  rechecking);  and 
(3)  maintenance  level  (depot,  field,  bench,  launch  pad,  flight  line, 
inflight). 

For  example,  the  SAIM  should  indicate  a  particular  interaction  between 
a  system  design  element  and  the  size  of  an  access  door.  The  other 
matrix  would  then  lead  the  maintainability  engineer  to  an  acceptable 
access  door  design.  This  matrix  would  not  show  directly  the  consequences 
to  weapon  system  effectiveness  of  violating  optimum  design  for  maintain¬ 
ability,  but  it  does  show  the  system  designers  that  a  problem  exists. 

This  is  a  trade-off  problem  and  maintainability  engineers  should  not 
attempt  to  restrict  the  trade-offs  to  the  human  factors  area. 

A  persistent  problem  is  the  frequent  inability  of  the  human  factors 
specialist  to  specify  how  much  and  even  what  kind  of  degradation  to  the 
weapon  system  will  follow  from  violating  optimum  maintainability  design 
features.  One  approach  is  to  consider  the  primary  criteria  of  good 
maintainability  (rapidity,  ease,  econony,  and  accuracy)  and  give  each 
a  qualitative  weight  for  every  item  in  a  maintainability  guide. 
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The  intent  of  the  matrix  and  scaling  approach  is  to  give  the  designer 
a  simple,  comprehensive,  and  consistent  framework  within  which  he  can 
organize  his  own  inputs.  "And  this  seems  to  he  a  basic  problem  with 
maintainability  today;  most  of  the  things  which  are  done  in  the  name  of 
maintainability  are  done  by  the  engineer  on  the  drawing  board.  This  is 
true  simply  because  for  every  maintenance  engineer  or  every  human 
engineer,  there  are  50  to  1500  designers . " 

Thus,  the  primary  value  of  interaction  matrices  is  that  they  can  point 
out  maintainability  problems.  All  of  the  solutions  may  not  be  known, 
but  it  is  still  important  to  identify  a  problem  and  then  provide  all 
the  currently  available  data  relevant  to  it.  One  difficulty  in  stating 
concrete  solutions  for  any  one  cell  defined  by  the  three-dimensional 
matrix  is  that  there  are  so  many  variables.  It  is  difficult  to  provide 
an  index  of  maintainability  information  that  contains  maintainability 
design  prescriptions  rather  than  principles.  Due  to  interaction  effects, 
it  may  not  even  be  feasible.  However,  the  design  guides  comnanly  used 
by  engineers  have  the  same  limitations,  e.g.,  the  discussion  on  tensile 
strength  of  steel  will  include  numbers,  graphs,  and  diagrams,  but  also 
will  say  that  tensile  strength  is  affected  by  heat.  The  three-dimensional 
matrix  on  maintainability  design  details,  though  written  at  a  cruder 
level  of  specificity,  will  be  similar  in  intent;  it  will  state  principles, 
or  even  formulas  where  they  exist,  and  it  will  point  out  interactions. 

There  is  a  need  for  a  maintainability  design  guide  for  the  systems  analyst 
and  design  engineer  that  tells  him  when  and  what  maintainability  design 
considerations  should  be  applied  to  the  design  of  a  weapon  system. 

Present  guides  present  the  total  body  of  things  that  ought  to  be  done 
with  regard  to  maintainability  design,  and  merely  urge  the  weapon  system 
designers  to  accomplish  them  as  early  as  possible.  A  system  guide  is 
needed  that  mates  procedures  used  in  developing  a  maintainable  system 
to  the  actual  milestones  in  system  development.  One  way  of  approaching 
this  is  to  list  the  decisions  made  in  the  design  process  that  have  a 
direct  influence  on  maintainability. 

In  order  to  be  effectively  implemented,  such  a  guide  would  have  to  be 
made  into  a  military  standard.  Present  day  maintainability  documents, 
no  matter  how  well  written,  are  read  by  only  a  few  design  personnel. 

It  might  be  better  (l)  to  have  the  weapon  system  require  maintaina¬ 
bility  design  in  accordance  with  a  certain  guide  or  group  of  guides 
specially  prepared  for  maintainability,  and  then  (2)  to  have  the  re¬ 
sponsibility  for  implementing  maintainability  design  given  to  a  special 
maintainability  group  rather  than  the  general  engineering  group.  At 
least  the  reliability  issue  has  been  successfully  handled  in  this  way. 

Maintainability  specialists  should  be  concerned  with  the  original  re¬ 
quirements  for  a  veapon  system  because  these  requirements  evolve  as 
desigi  work  proceeds.  At  first,  one  cannot  ordinarily  state  clean-cut 
requirements,  such  as  how  much  time  is  available  for  maintenance.  The 
maintainability  group  can  be  of  great  assistance  in  setting  tentative 
requirements  and  successively  refining  them. 


11 


An  engineer  does  not  assemble  an  electrical  circuit  to  perform  a  given 
function  by  picking  components  at  random  and  then  testing  to  see  if  it 
does  perform  the  function.  He  has  a  scheme  in  mind  that  presumably 
gives  him  a  high  probability  of  coming  up  vith  a  circuit  that  will 
perform  this  function.  This  same  kind  of  scheme,  and  this  same  kind  of 
"expectation,"  is  needed  for  maintainability  design  also. 

One  reason  for  instituting  a  maintainability  group  is  that  there  are  so 
few  maintainability  people  per  design  engineer.  A  separate  group  is 
required  to  help  introduce  inputs  to  the  designers  who  have  no  background 
in  maintainability.  Another  reason  for  establishing  a  maintainability 
group  is  that  the  average  design  engineer  does  not  use  the  guides 
available  today,  and  there  is  no  assurance  that  he  would  use  guides 
even  if  they  were  better.  Some  central  source  to  push  maintainability 
implementation  appears  essential. 

Evaluation  of  maintainability  design  appears  to  be  unexploited.  The 
traditional  problems  of  criterion  development  can  be  expected:  relia- 
bility  of  measurement,  combination  of  partly  correlated  indexes,  etc. 

A  major  problem  is  that  the  usual  criteria  are  far  too  broad:  the  need 
is  for  proximal  criteria,  not  distal.  Perhaps  it  is  going  too  fax  to 
try  to  relate  the  effects  of  a  particular  accessibility  design  to  the 
ultimate  performance  of  the  weapon  system. 

8.  Department  of  the  Air  Force,  Maintainability  Program  for  Weapon,  Support, 
and  Command  and  Control  Systems,  AF  Regulation  No.  66-29,  November  22, 

iwr. - 

"This  regulation  states  the  policy,  responsibility,  and  basic  procedures 
for  the  USAF  Maintainability  Program  for  weapon,  support,  and  conmand 
and  control  systems.  It  applies  to  Air  Force  activities  engaged  in 
research,  development,  testing,  procurement,  maintenance,  and  operation 
of  new  Air  Force  systems,  subsystems,  and  components . " 

9.  Fellows,  J.  L.,  Jr.,  The  Human  Factors  Engineering  Aspects  of  Maintenance, 
Fifth  Annual  U.S.  Aimy'l'ul&n  Factors  Engineering  Conference,  Redstone 
Arsenal,  September,  1959#  pp.  69-70. 

"Space  restrictions  placed  on  the  battlefield  by  the  threat  of  atomic 
warfare  will  seriously  limit  the  number  of  people  we  can  devote  to 
maintenance,  and  as  a  result  limit  the  time  we  can  spend  to  perform 
maintenance  at  the  organizational  level."  (it  seems  reasonable  that 
this  restriction  applies  also  to  certain  Air  Force  activities.)  "This 
means  that  me  must  develop  the  state-of-the-art  to  meet  these  restrictions. 
We  cannot  tolerate  any  extra  units,  or  extra  people,  or  extra  equipment 
to  perform  maintenance,  which  can  be  eliminated  by  adherence  to  the 
principles  of  human  factors  engineering." 
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The  following  goals  should  be  considered  in  equipment  design: 

1.  Improve  accessibility  for  adjustment  and  minor  repair 
by  the  operator. 

2.  Reduce  time  required  to  perform  this  adjustment  and 
repair. 

3*  Reduce  technical  skill  and  manual  dexterity  required 
for  adjustment  and  both  minor  and  major  repair. 

k.  Reduce  number  of  special  tools  and  test  equipment 
needed  in  the  field. 

5.  Reduce  checkout  in  the  forward  areas  to  simple  go/ 
no-go  checks  with  quick  plug-in  assembly  replacements. 

6.  Design  assemblies  and  subassemblies  to  permit  economical 
throw-away  of  faulty  items  instead  of  costly  repair. 

7.  Design  automatic  test  equipment  for  use  in  rear  areas  for 
expensive  assemblies  that  must  be  repaired  Instead  of 
thrown  away. 


10.  Firstman,  S.  I.,  M.  Kamins,  and  B.  J.  Voosen,  Automatic  Checkout 

Equipment:  Employment  and  Design  Considerations,  The  RAND  Corporation, 
Report  R-3587  September,  i960,  78  pp. 

This  report  considers  six  design  choices  that  arise  in  the  development 
of  nearly  any  checkout  system: 

1.  Internal  and  external  programming 

2.  Open-loop  and  closed-loop  testing 

3.  Static  and  dynamic  testing 

k.  Analog  and  digital  measurement  methods 

5.  qualitative  and  quantitative  testing 

6.  Integral  and  auxiliary  test  equipment 

Alternatives  are  examined  on  the  basis  of  cost,  speed  requirements, 
availability,  flexibility,  and  accuracy.  In  none  of  the  six  Issues 
does  it  appear  that  a  particular  alternative  is  universally  favored. 
Trade-off  relations  (e.g.,  cost  versus  accuracy)  are  hypothesized  for 
some  of  the  alternative  configurations. 

While  it  appears  impossible  to  formulate  a  comprehensive  set  of  rules 
that  can  render  an  "automatic"  evaluation  of  particular  ACE  designs. 
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the  Investigation  did  reveal  guidelines  that  remain  standard  over  a 
wide  range  of  circumstances.  Thus,  analog  systems  are  usually  cheaper 
if  accuracy  requirements  are  not  too  stringent.  Qualitative  indications 
are  often  insufficient  for  estimating  "confidence”  in  system  state;  if 
analog  computers  are  employed  as  feedback  elements,  then  self -test  may 
prove  difficult  or  expensive. 

"No  clear-cut  preferred  method  emerges  as  a  universal  solution  to  all 
problems,  or  even  to  any  broad  class  of  problems,  in  this  field.  The 
weapon-system  character! sties  and  support  requirements;  the  tester's 
required  speed,  accuracy,  and  reliability  characteristics  for  each 
function  of  the  system;  and,  most  important,  operational  considerations, 
all  will  affect  the  composite  and  will  often  present  conflicts.  Each 
design  is  likely  to  involve  a  compromise  among  various  considerations. 
Rarely,  if  ever,  will  all  considerations  lead  to  the  same  method  or 
unique  combination  of  methods." 


11.  Jblley,  J.  D.,  Jr.,  Maintenance  Job  Simplification  Without  Automation, 
American  Institute  for  Research,  Memorandum  No.  lU,  September,  1959, 

7  PP- 

An  inverse  relationship  is  postulated  between  equipment  complexity  and 
the  quality  of  maintenance.  Solutions  involving  ACE  and  increased 
training  time  have  not  proven  entirely  satisfactory.  Use  of  ACE  has 
created  new  problems — maintenance  of  the  test  gear  itself,  increased 
cost,  logistics,  longer  lag  in  equipment  development,  and  increase  in 
over-all  system  size.  . 

Troubleshooting  is  the  main  problem;  one  possible  solution  entails  the 
provision  of  complete  instructions  for  the  full  range  of  troubleshooting 
problems  to  be  encountered. 

In  1933  Dr.  Robert  B.  Miller  made  practical  suggestions  for  the  con¬ 
tent  and  format  of  Job  manuals  to  simplify  the  training  problem. 

Also  in  that  year  Miller,  Folley,  and  Smith  suggested  the  "half-split 
troubleshooting  technique."  Since  that  time,  a  number  of  studies 
have  demonstrated  that  fully  routine  troubleshooting  procedures  are 
feasible  and  can  be  applied  by  unskilled  persons  to  troubleshoot  com¬ 
plex  equipment. 

The  difficulty  with  such  complete  troubleshooting  procedures  is  that 
they  are  cumbersome.  Many  pages  and  contingency  prescriptions  must  be 
provided.  Research  is  needed  to  establish  the  optimum  degree  of 
routinization  for  manual  troubleshooting  procedures.  It  is  speculated 
that  maximum  maintenance  efficiency  is  greatest  when  the  procedures  are 
near  the  middle  between  the  two  extremes  of  "no  procedure,  all  decisions" 
and  "all  procedures,  no  decisions." 

"1.  The  entire  troubleshooting  task  need  not  be  completely 

procedural.  In  fact,  zero-decision  procedures  are  probably 
more  cumbersome  than  necessary. 
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2.  Troubleshooting  aids  start  with  some  kind  of  systematic 
check  of  the  system,  or  portion  of  the  system.  Do  not 
try  to  start  with  haphazard  malfunction  indications  that 
you  can  think  of. 

3.  Indications  at  any  step  in  the  checkout  procedure  or 
troubleshooting  procedure  can  be  dichotomized  into  a 
specified  indication  on  the  one  hand,  or  'anything  else' 

on  the  other.  This  makes  an  easy-to-follow  two-alternative 
choice  at  each  step. 

4.  You  can  make  any  assumptions  that  are  appropriate  about 
the  skill  level  of  the  technician  and  pitch  your  trouble¬ 
shooting  procedure  at  that  level.  Whether  the  technician 
be  a  novice  or  an  expert,  some  hints  regarding  the  pro¬ 
cedure  to  follow  will  probably  be  helpful. 

"Remember  —  if  you  cannot  prepare  some  kind  of  a  troubleshooting  pro¬ 
cedure,  or  your  design  engineers  say  it  is  too  complicated  to  be  done, 
you  are  hiding  your  head  in  the  sand  with  regard  to  the  maintenance  of 
your  system.  If  the  designers  cannot  tell  how  to  troubleshoot  the 
system,  how  can  you  realistically  expect  the  maintenance  technician  to 
figure  it  out?" 


12.  Franks,  P.  E.,  and  C.  W.  Furnish,  Automated  Maintenance:  Theory, 

Practice,  and  Implications  for  Training,  WADD  Technical  Report  o0-4l2, 
August,  i960,  23  pp. 

The  automatic  checkout  concept  may  be  justified  insofar  as  it  (l) 
reduces  skilled  manpower  needs,  or  (2)  serves  to  achieve  an  extremely 
short  operational  reaction  time.  A  successful  checkout  system  "must 
be  multipurpose,  accurate,  efficient  (in  terms  of  testing  time),  seldom 
fail,  and  have  a  quick  change  capability  while  keeping  costs  within 
reasonable  limits." 

A  developmental  trend  toward  more  "universal"  testers  is  indicated. 

Thus,  any  checkout  system  can  be  described  in  terms  of  four  basic  units: 
(l)  the  instruction  center  or  programmer;  (2)  the  stimuli  subsystem  or 
signal  generator;  (3)  The  data  summary  subsystem  or  comparator;  and  (4) 
the  self-test  subsystem  which  monitors  the  checkout  system  itself. 

Within  each  of  these  four  units,  one  can  delineate  more  detailed 
functions  (e.g.,  interrogator,  timer,  digitizer,  etc.)  that  are  still 
not  specific  to  a  given  system.  The  performance  limitations  of  an 
automatic  system  are  likely  to  include  intermittent  failures,  waveform 
comparisons,  mechanical  checkouts,  and  adjustment  of  out-of-tolerance 
components . 

Functional  tests,  conducted  with  the  equipment  in  its  normal  environment 
and  operating  state,  are  distinguished  from  marginal  tests,  which  push 
system  parameters  to  or  beyond  operational  extremes.  Marginal  tests 
may  be  useful  in  predicting  incipient  failures  in  a  large  collection  of 
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parallel,  similar  circuit  elements,  since  only  a  few  parametric 
excursions  (e.g.,  heater  and  screen  voltages)  are  required  for  checking 
all  the  components.  Rmctional  tests  are  most  practical  for  subsystems 
that  are  highly  variable  and  arranged  in  series. 

Over-all  systems  checks  are  favored  over  component  tests  because  they 
"include  tests  of  internal  cabling  and  critical  matching  of  major  sub¬ 
systems."  Component  tests  may  be  good  for  automatic  troubleshooting 
and  are  "especially  useful  with  isolated  networks  of  passive  components, 
cable  and  harness  assemblies,  etc." 

Both  static  and  dynamic  tests  may  be  programmed  into  an  ACE  sequence. 

In  a  static  test,  a  program  of  intermittent  sigaals  is  applied,  and 
outputs  are  allowed  to  reach  a  steady  state  before  comparison.  The 
dynamic  test  simulates  a  continuous  input  and  exercises  the  system 
under  more  realistic  conditions.  Dynamic  tests  sure  generally  more 
difficult  to  instrument  and  evaluate,  but  the  information  gained  has 
direct  operational  significance. 

Closed-loop  testing,  though  essential  for  engineering  evaluation,  is 
not  usually  satisfactory  for  field  ACE  applications  since  it  introduces 
additional  equipment  complexity  and  its  intrinsic  self-correcting 
features  tend  to  suppress  cumulative  performance  trends.  In  contrast, 
the  open-loop  test  is  simpler,  cheaper,  and  more  directly  indicative 
of  over-all  system  performance.  "Prom  present  experience  in  military 
field  operations  it  is  believed  that  the  open  loop,  dynamic  system  test 
is  the  best  test  exercise  in  the  sense  that  it  most  frequently  provides 
an  optimum  tradeoff  between  cost  and  complexity  versus  system  reliability. 
In  same  cases,  however,  a  static  test  should  be  considered  as  it  may 
permit  considerable  savings  in  hardware  with  little  detriment  in  test 
quality." 

What  tasks  should  be  left  for  the  technician?  These  will  depend  on  the 
system  to  be  checked  and  the  amount  of  information  desired.  An  illustra¬ 
tive  checkout  system  (for  the  A3J  Naval  aircraft )  prescribes  the  following 
manual  operations: 

1.  Make  cable  connections 

2.  Make  initial  settings  in  test  unit 

3.  Use  the  instruction  book  for  interpretation 

It.  Monitor  and  analyze  self -test  results 

5-  Make  replacements 

6.  Make  adjustments  to  prime  equipment 

Assuming  that  ACE  will  soon  be  in  use  and  time  is  short  to  prepare  for 
its  Impact,  the  anticipated  effects  of  ACE  on  personnel  skill  levels, 
motivation,  logistics,  cost,  system  accuracy,  and  training  are  discussed. 
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Some  of  these  affects  are  likely  to  be  complex  and  subtle;  e.g., 
automated  checkout  may  actually  raise  average  skill  levels  rather  than 
lover  then,  but  the  requirements  for  extremely  high  skill  levels  may 
be  sharply  reduced.  Increased  emphasis  on  ACE  in  existing  training 
courses  is  recoananded;  this  vill  provide  backup  if  ACE  does  not  prove 
to  be  as  reliable  as  anticipated. 


13.  Gibson,  W.  R.,  "What  Price  Maintainability,"  in  F.  L.  Ankenbrandt  (ed. ), 
Electronic  Maintainability,  Vol.  3>  Reinhold,  New  York,  i960. 

"Good  maintainability  is  a  highly  beneficial  equipment  attribute.  How¬ 
ever,  the  allowable  expenditures  of  manpower  and  time  to  achieve  this 
goal  must  be  kept  within  definable  limits.  These  expenditures  must  be 
identified,  assigned  a  value,  and  evaluated  with  relation  to  the  net 
gain  to  be  realized  from  each  maintainability  reconmendatlon." 

A  system  for  estimating  trade-offs  between  maintainability  costs  and 
gains  is  described:  When  the  mad  n  tain  ability  engineer  makes  a  recom¬ 
mendation,  it  is  examined  according  to  the  resulting  net  change.  A 
work  sheet  for  recording  trade-off  estimates  is  reproduced,  and  sample 
entries  sure  described.  Many  factors  are  recorded  for  each  design 
recommendation;  e.g.,  "customer"  gains  or  losses,  such  as  scheduled 
servicing  demands  and  mean-time-between-failures. 


14.  Hirshon,  s.,  "The  Air  Force  Outlook  on  Maintainability,"  in  F.  L. 

Ankenbrandt  (ed. ),  Electronic  Maintainability,  Vol.  3,  Reinhold,  New 
York,  i960. 

"The  Air  Farce  must  move  forward  with  industry  to  develop  the  philosophy 
and  methods  for  the  production  of  more  maintainable  weapon  systems. 

Again  we  will  repeat  that  the  establishment  of  maintainability  in 
meaningful  terms  is  necessary  to  establish  the  benchmarks  for  sound 
engineering  Judgnents  to  achieve  maximum  weapons  value.  Maintainability 
is  a  major  design  parameter  and  is  a  measure  of  weapon  system  effective¬ 
ness.  Therefore,  all  its  significant  aspects  must  be  identified, 
specified,  measured,  and  controlled  to  produce  weapon  systems  which  can 
be  most  economically  maintained  in  a  condition  of  readiness. 

"We  visualize  completely  automatic  sensing  devices  which  when  coupled 
with  built-in  redundant  circuits  will  rapidly  detect  the  faulty  unit 
and  revert  to  the  additional  operating  mode  for  safe  operation.  We 
visualize  automatic  self-calibrating  and  self-regulating  equipment  to 
insure  constant  in-tolerance  operation  coupled  with  warning  devices  to 
indicate  when  out-of-tolerances  cannot  be  self -corrected.  We  visualize 
built-in  margined,  stressing  circuits  which  will  predict  incipient 
failures  and  indicate  the  need  for  preventative  maintenance.  We  visualize 
all  low  and  medium-power  and  most  high-power  electronic  circuitry  handled 
by  solid-state  modules  one-twentieth  to  one-fiftieth  the  size  of  present- 
day  conventional  units  performing  the  same  functions.  Such  modules 
because  of  their  high  density  and  solid-state  construction,  will  not  be 
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amenable  to  economic  repair.  We  therefore  visualize  extensive  use  of 
disposal  at  failure  maintenance  rather  than  repair* type  maintenance. 
The  designers  Incorporating  disposal  at  failure -type  modules  coupled 
with  built-in  fault  locating  devices  will  largely  eliminate  the  need 
for  highly  trained  technicians  for  repair." 


15.  Holaban,  J. ,  "Direction  and  Better  Engineering  Needed  for  Automatic 
Checkout,"  Space/Aeronautics,  Vol.  34,  No.  4,  October,  i960,  pp. 
210-214. 


The  article  states  that  ACE  is  a  confused  field.  Part  of  the  confusion 
is  due  to  the  manufacturers,  "both  by  (their)  poor  engineering  and  by 
(their)  grossly  overstating  the  capabilities  of  their  products.  The 
sales  claims  of  the  average  checkout  equipment  company  are  at  least 
five  years  ahead  of  engineering  performance."  As  a  rejoinder,  the 
manufacturers  can  claim  that  the  military  doesn't  have  a  firm  maintenance 
policy.  A  case  In  point,  they  say,  is  the  military' s  wavering  attitude 
toward  throwaway  modules.  "If  ever  put  into  practice,  it  would  un¬ 
doubtedly  simplify  checkout  design  and,  in  fact,  the  entire  maintenance 
problem.  But .. .because  these  decisions  haven't  been  made,  checkout  gear 
today  must  be  designed  to  localize  faults  down  to  as  close  to  the  piece 
part  as  possible." 

Several  persistent  problems  of  ACE  development  are  noted:  (l)  the 
desire  for  too  much  versatility  for  the  ACE;  (2)  the  effects  of  changes 
in  prime  equipment  on  ACE  design;  (3)  the  attempt  to  automate  existing 
manual  procedures. 

Eight  functions  are  performed  by  most  ACE  designs:  programming,  signal 
stimulation,  comparison,  conversion,  evaluation,  readout,  memory,  and 
switching.  "It's  hard  to  believe  that,  even  with  much  more  sophisticated 
equipment,  man  will  ever  be  completely  removed  from  checkout."  Among 
the  functions  left  to  man  is  his  ability  to  "understand  the  peculiarities 
of  the  machine  in  a  variety  of  situations  and  allow  for  them." 

Can  a  universal  test  set  be  built?  The  answer  depends  on  the  meaning 
of  universality,  but  systems  such  as  the  Kortronics  Datico  and  the  RCA 
Dee  can  test  a  great  majority  of  the  functions  of  present  missiles  and 
electronics  systems. 

Since  ACE  itself  is  not  completely  reliable,  the  question  of  "who  checks 
the  checker"  is  often  raised.  This  question  does  not  necessarily  lead 
to  an  infinite  regress  of  equipment  proliferation,  and  there  is  already 
evidence  that  proper  design  (e.g.,  the  Amy's  Fait  System)  can  "check 
the  checker"  in  an  efficient  way. 

A  trend  is  foreseen  away  from  the  present  serially-programmed  equipment, 
to  parallel-programmed  tests  under  central  computer  control.  Among  the 
advantages  claimed  for  parallel  diagnostic  tests  are  faster  checking, 
less  severe  test  point  constraints,  easier  marginal  detection  in  complex 
networks,  and  the  fact  that  refinements,  such  as  life  prediction  and 
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parts  ordering,  can  be  built  in.  Arguments  against  the  parallel  concept 
are  the  complex  computer  program  required  and  generally  high  costs.  For 
the  short-term  future,  "incremental"  gains  in  ACE  engineering  are  an¬ 
ticipated:  higher  speed,  greater  accuracy,  more  test  points,  and  more 
test  storage. 


16.  Imus,  H.  0.,  "Tomorrow's  Quality  Demands,"  IRE  Trans.  Reliability  and 
Quality  Control,  PGRQC-13,  July,  1958* 

The  author  identifies  some  general  factory  automation  trends  and  their 
consequences  on  personnel,  complexity,  standardization,  precision,  and 
speed.  He  identifies  and  discusses  specific  consequences  of  automation 
on  quality  control,  personnel,  precision,  and  production  cycle  time. 

Quality  Control.  "Test  and  inspection  equipment  will  be  mechanized  and 
mechanically  integrated  into  the  production  system.  This  is  part  of 
the  ultimate  overall  automation  philosophy  as  discussed  previously. 

The  test  and  inspection  equipment  of  today  is  almost  completely  manual 
or  only  partially  mechanized,  comparatively  far  behind  production 
equipment.  There  is,  however,  equipment  either  on  the  drawing  board 
or  in  the  market  that  is  fully  mechanized. 

"Statistical  analysis  will  be  built  into ...  automated  inspection  equipment. 
Information  obtained  by  the  inspection  equipment  will  be  fed  back  into 
the  machine  computer.  The  computer  controls  the  position  of  the  tool, 
compensating  for  tool  wear  and  machine  drift. 

" Post -production  inspection  is  conmon  today.  Automation  will  emphasize 
in-production  find  pre-production  inspection .  In-production  Inspection  . 
will  provide  quality  information  while  the  part  is  actually  being 
produced. 

"One  hundred  per  cent  inspection  will  become  more  widespread.  In  many 
cases  automated  equipment  to  inspect  output  100  per  cent  will  cost  only 
a  little  more  than  equipment  to  inspect  10  per  cent. 

"The  most  critical  quality  control  activity  will  take  place  prior  to 
actual  production.  Decisions  will  be  made  at  that  time  to  fix  the 
level  of  quality.  New  design  control  will  be  a  must  in  the  automated 
factory  as  automatic  equipment  tends  not  to  be  flexible." 

Personnel.  "In  the  automated  production  line  the  human  factor  is 
minimizes,  and  also  the  importance  of  quality -mindedness .  Therefore, 
quail ty-odndedness  must  be  emphasized  with  engineering  and  planning 
personnel.  The  personnel  dealing  with  automation  will  of  necessity 
have  higher  qualifications  than  today's  hand-assembly  factory  laborer 
....Inspectors,  as  such.  Just  won't  exist  and  in  their  place  will  be 
highly  skilled  inspection-test  equipment  setup  and  maintenance 
personnel.  Test  and  inspection  planning  will  be  more  elaborate." 
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Precision.  "With  the  advent  of  products  demanding  and  machines  capable 
of  producing  parts  to  tolerances  in  the  tens  of  thousandths  and  micro- 
* 

inches,  the  quality  control  function  must  make  adequate  provision  to 
match  these  tolerances." 

Production  Cycle  Time.  "Customers  are. . .demanding  that  we  substantially 
reduce  tSe  production  cycle  time.  With  the  time  squeeze  on  all  functions, 
it  is  well  that  all  quality  control  objectives,  methods,  and  responsi¬ 
bilities  be  recognized  and  integrated  well  into  the  pattern  of  a  fast 
operation." 


17.  Latta,  W.  B.,  "The  Array  Outlook  on  Maintainability,"  in  F.  L.  Ankenbrandt 
(ed. ),  Electronic  Maintainability,  Vol.  3,  Reinhold,  New  York,  i960. 

Maintainability  is  a  "two-edged  sword  of  management . "  One  edge  relates 
to  the  reduction  of  logistics  requirements;  the  other  treats  maintaina¬ 
bility  as  an  aid  to  "decisions  concerning  our  equipment  inventory  as  it 
rapidly  obsolesces." 

The  Amy's  Review  Point  Program  directs  attention  to  maintainability 
throughout  the  procurement  cycle.  A  continuing  problem  is  adequate 
definition:  "From  where  I  sit  in  the  Pentagon,  it  may  take  ten  years 
or  even  a  second  ten  years  to  arrive  at  our  goal  of  measurable  contractual 
language  to  define  ease  of  maintenance  in  Army  equipment .  There  are  a 
number  of  equally  fundamental  answers  we  must  obtain  in  order  to  measure 
maintainability."  Objective  functions  are  desirable,  but  have  not  yet 
been  derived,  for  the  various  costs  involved  in  keeping  a  repair  part 
in  the  system,  for  repair  costs  by  the  technician,  and  for  the  throw¬ 
away  module  policy. 

A  brief  history  of  contract  effort  on  the  AN/GRC-66  Radio  Relay  is 
given  as  an  example  of  a  maintenance-oriented  contract.  On  the  basis 
of  field  experience  with  an  earlier  model  (AN/TRC-29),  the  number  of 
circuits  was  reduced;  the  failures  to  be  expected  in  the  new  system 
are  largely  limited  to  modules  that  can  be  stocked  in  forward  echelons. 

Many  variables  affect  equipment  obsolescence;  e.g.,  class  of  equipment, 
elapsed  time  since  the  item  was  developed,  rate  of  technological  advance, 
budgetary  and  production  resources,  and  operational  criteria.  Obso¬ 
lescence  curves  are  related  to  calendar  time,  maintenance-required- to- 
re store,  and  reliability.  An  "index  of  competitiveness"  is  proposed  to 
measure  the  worth  of  retaining  equipment  items  in  the  system.  The  index 
is  now  being  applied  to  four  types  of  electronics  equipment. 


The  Inchworm  Motor,  manufactured  by  the  Airborne  Instruments  Laboratory, 
Mineola,  Long  Island,  New  York,  controls  certain  metalworking  machines  to 
hold  dimensions  to  plus  or  minus  five  micro-inches." 
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18.  Lewis,  J.  L. ,  "Integration  of  Maintainability  Requirements  into  the 

Research  and  Development  Phase  of  Weapon  Systems,"  in  F.  L.  Ankenbrandt 
(ed. ),  Electronic  Maintainability,  Vol.  3,  Reinhold,  New  York,  i960. 

Because  of  budgetary  limits,  Increasing  complexity,  and  short  military 
reaction  times,  "it  is  unlikely  that  ve  will  ever  again  have  more  men 
or  material  than  we  need  to  do  a  job." 

"In  stannary,  what  is  needed  from  the  Air  Force  point  of  view  is  an 
adequate  and  integrated  program  to  implement  the  requirement  that 
weapon  system  components  be  serviceable  and  repairable.  This  effort 
must  begin  at  the  time  equipment  design  begins....  The  philosophy  of 
the  maintenance  concept  that  is  to  be  used  in  a  specific  system  must 
be  developed  in  its  generic  entirety,  as  far  down  in  specific  detail 
as  can  reasonably  be  described,  in  the  planning  phase." 


19 .  Manley,  R.  W.,  "Management  Techniques  for  Achieving  Maintainability," 
in  F.  L.  Ankenbrandt  (ed.).  Electronic  Maintainability,  Vol.  3# 

Reinhold,  New  York,  i960.  ~ 

A  discussion  is  presented  of  "roadblocks  to  maintainability"  that  must 
be  overcome  "by  any  organization  which  is  to  be  successful  in  desiring, 
developing,  and  manufacturing  maintainable  electronic  hardware."  These 
factors  Include  disinterest  of  designers,  unsuitable  organizational 
structure,  lack  of  design  review  system,  and  mishandling  of  field 
information. 

The  techniques  adopted  for  assuring  combat  readiness  (and,  therefore, 
maintainability)  of  the  Falcon  missiles  are  described.  One  of  these 
techniques  is  to  set  aside  one  test  model  for  maintainability  study; 
this  missile  is  completely  tested,  disassembled,  and  reassembled  with 
field  maintenance  tools.  Careful  records  are  kept  of  tests,  repair 
parts,  and  other  facts  of  maintainability  significance. 


20.  Miller,  R.  B.,  Anticipating  Tomorrow' s  Maintenance  Job,  USAF  Air 
Training  Coasnand,  Research  Review  53-1#  March,  1953#  21  pp. 

An  "equipment -demands"  analysis  conducted  on  a  prototype  equipment  can 
be  used  to  predict  requirements  for  the  production  equipment.  A  sample 
application  of  the  method  to  the  Q-24  radar  showed  a  remarkable  corre¬ 
spondence  between  the  prototype  and  production  models.  Some  practical 
implications  sure  that  (1)  mechanics  can  be  trained  before  equipment  i6 
obtained,  (2)  basic  or  "core"  courses  can  be  set  up  for  related  but 
different  Jobs  via  common  job-derived  activities,  (3)  technical  manuals 
can  be  written  in  behavioral  language,  and  (4)  Job  design  can  proceed 
simultaneously  with  the  early  stages  of  equipment  design. 

21.  Ozkaptan,  Halim,  "Maintenance  Problems  of  Space  Flight,  "  in  F.  L. 
Ankenbrandt  (ed. ),  Electronic  Maintainability,  Vol.  3#  Reinhold,  New 
York,  i960. 
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The  unique  features  of  maintenance  in  space  are  pointed  out:  logistics, 
"fallback"  resources,  special  planning,  and  psychological  problems  such 
as  weightlessness,  confinement,  and  time  criticality.  Troubleshooting, 
the  major  factor  in  repair  time,  is  postulated  as  the  area  in  which 
the  greatest  returns  relative  to  support  effort  can  be  achieved. 

Past  efforts  to  aid  the  troubleshooter  have  usually  taken  the  form  of 
special  manuals,  equipment  layout,  and  test  point  selection.  "However, 
these  have  only  served  as  aids  to  the  technicians'  capability  without 
substantially  increasing  it.  A  more  significant  point  is  that  they 
also  require  decision  and  application  in  themselves.  Voluminous 
manuals  present  a  task  in  their  application  which  is  almost  as  formidable 
as  the  troubleshooting  problem.  Similarly,  training  in  basic  electronics 
does  not  transfer  directly  to  the  troubleshooting  process.  Fbr  space 
and  other  remote  complex  systems,  a  direct  amplification  of  trouble¬ 
shooting  capacity  is  required  to  keep  pace  with  increasingly  complex 
systems  and  decreasing  repair  time  available.  In  other  words,  since 
the  scope  of  equipment  is  surpassing  man's  ability  to  diagnose  it, 
equipment  must  be  used  to  amplify  this  capacity  rather  than  supplement 
it." 


How  is  this  amplification  to  be  achieved?  "In  the  past  we  have  amplified 
human  performance  in  our  advanced  systems  through  such  means  as  servo¬ 
mechanisms  and  special  displays.  For  troubleshooting,  a  similar  approach 
can  be  employed  relative  to  an  optimum  continuation  of  human  and 
equipment  abilities.  Computers  which  have  greater  rote  memory  and  speed 
of  association  than  humans  can  be  used  effectively  in  combination  with 
man's  greater  power  of  sensory  discrimination  and  flexibility.  The 
components  of  a  system  and  their  relationships  can  be  programmed  into 
a  ground-based  computer.  Depending  on  the  amount  of  failure  information 
or  symptoms  which  can  be  determined  and  remotely  fed  by  the  astronaut 
to  the  computer,  it  can  run  trial  solutions  and  either  idenfity  the 
failed  component,  give  decreasing  probabilities  for  the  failed  component, 
or  indicate  additional  information  needed.  This  would  represent  computer 
amplification  of  the  troubleshooting  process  Itself,  supplementary  to 
the  capability  of  the  maintenance  technician." 

A  central  computer  installation  could  be  set  up  to  service  a  large 
area  and  many  different  systems.  Data-link  interrogation  of  the  central 
computer  would  permit  distant  but  rapid  troubleshooting,  since  the  remote 
technician  could  send  in  masses  of  test  indications  and  the  computer 
could  send  back  preliminary  interpretations  and  guides  to  further  checking. 

For  such  an  arrangement  to  work,  the  optimum  configuration  of  man  and 
computer  must  be  set  up.  The  computer  might  be  programmed  to  store  all 
imaginable  faults,  with  each  related  to  a  pattern  of  symptoms .  A  more 
sophisticated  approach  would  involve  a  dynamic  model  of  the  internal 
characteristics  of  the  system,  along  with  "simulation  runs"  that  depend 
on  logical  relationships  to  indicate  probable  failed  units .  This  second 
method  is  the  most  appealing,  but  would  require  much  development  effort. 

Some  problems  in  man-computer  communication  and  in  the  implementation 
of  the  maintenance  center  are  briefly  examined.  Potential  advantages 
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of  the  center  include  preventive  maintenance  programming,  inventory 
and  logistics  tabulations,  and  the  collection  of  reliability  statistics. 

If  the  concept  is  successful,  the  training  and  skill  levels  of  technicians 
might  be  materially  reduced;  at  the  same  time,  troubleshooting  aids  and 
manuals  could  be  simplified. 


22.  Steele,  G.,  and  P.  Kircher,  The  Crisis  We  Face,  McGraw-Hill,  New  York, 

I960,  220  pp. 

This  book  identifies  the  management  and  design  problems  that  accompany 
the  use  of  automatic  data  processing  systems.  Some  "horrible  examples" 
are  described  in  which  missile  programs  have  been  victims  of  an  imbalanced 
and  unrealistic  attitude  toward  automatic  testing.  However,  the  authors 
foresee  a  tremendous  advance  provided  management  realizes  that  techno¬ 
logical  changes  are  occurring.  Many  suggestions  are  given  for  professional 
management  training;  one  of  these  proposes  a  seven-year  college  course 
emphasizing  the  engineering  and  data  processing  sciences. 


23.  Still,  W.  L.,  "A  Rebuttal  by  the  Military,"  Control  Engineering,  Vol.  f. 
No.  12,  December,  i960,  pp.  77-63.  ~B~ 

This  article  is  a  vigorous  refutation  of  Steele  and  Kircher’ s  The  Crisis 
We  Face  (see  Item  22).  It  is  contended  that  the  authors'  extrapolation 
from  Atlas  to  Minuteman  is  faulty,  and  that  the  Minuteman  actually 
represents  a  considerable  simplification  in  equipment  and  procedure . 

For  example,  "Conanand  and  control  of  the  (Minuteman)  missile  originates 
in  the  Launch  Control  Center,  the  manned  post  from  which  two  men  govern 
all  missile  functions.  The  message  structure  to  control  the  automatic 
force  consists  of  a  total  of  six  comnands  transmitted  from  the  Launch 
Control  Centers."  As  for  the  data  fed  back  from  the  launch  site,  "less 
than  a  dozen  discrete  status  signals  are  returned  from  the  missile." 

The  targeting  procedure  hypothesized  by  the  authors  for  the  Minuteman 
is  also  criticized. 

Steele  and  Kircher' s  analyses  of  Minuteman  component  parts  and  relia¬ 
bilities  are  questioned  on  both  numerical  and  logical  grounds.  A  new 
Minuteman  approach  to  treatment  of  failure  data  is  described  in  which 
each  failed  component  is  sent  to  a  "Physics  of  Failure  Laboratory."  It 
is  maintained  that  "by  taking  all  possible  known  preventive  steps  from 
initial  component  manufacture,  and  operating  on  the  premise  that  there 
is  no  such  thing  as  a  random  failure,  we  can  gradually  improve  manu¬ 
facturing,  engineering,  handling,  and  operating  procedures  to  approach 
our  reliability  goals." 


24.  Swain,  A.  D.,  and  J.  G.  WOhl,  Factors  Affecting  Degree  of  Automatic ity 
in  Teat  and  Checkout  Equipment,  Dunlap  and  Associates,  Inc.,  StamfordT 
Conn.,  TR  60-3 6F,  March,  1961. 
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A  survey  of  problems  in  the  design  and  use  of  automatic  checkout 
equipment  was  directed  at  specifying  both  formal  and  informal  factors 
that  affect  (l)  the  degree  of  automaticity  designed  into  test  and 
checkout  operations,  and  (2)  the  field  acceptability  of  ACE.  Interviews 
were  conducted  with  design  and  research  personnel  in  the  areas  of  prime 
equipment  and  test  equipment  design,  maintenance  training,  and  human 
factors.  Interviews  and  observations  were  conducted  at  a  number  of 
operating  military  sites.  A  literature  search  was  made  of  reports  on 
design  and  experience  in  the  areas  of  maintenance  testing. 

On  the  basis  of  the  survey,  the  report  describes  current  allocation  of 
test  and  checkout  operations  between  man  and  machine,  and  shows  the 
implications  for  the  effectiveness  or  ineffectiveness  of  field  operations 
where  test  and  checkout  equipment  is  used.  Overriding  (or  "super") 
factors  and  compromises  affecting  ACE  implementation  are  emphasized. 

Recommendations  are  made  for  a  systematic  allocation  procedure,  and 
some  allocation  guidelines  are  presented.  These  guidelines  vary  from 
discussions  of  specific  weapon  systems  and  specific  criteria  to  a  more 
general  treatment  of  those  tasks  that  man  or  machine  does  best,  and 
the  various  conditions  under  which  the  superiority  of  each  is  reduced  or 
reversed.  Finally,  suggestions  are  made  for  design  features  that  can 
improve  the  compatibility  of  ACE  with  its  operating  environment,  and 
for  a  more  efficient  role  of  man  in  the  maintenance  process. 


25.  Toffler,  J.  E.,  "Some  Typical  Problems  in  the  Design  of  Automatic  Test 
Equipment,"  Electronic  Industries,  Vol.  20,  No.  1,  January,  1961, 
pp.  76-80. 

Problems  in  designing  a  high-speed,  automatic  system  to  be  used  for 
missile  checkout  by  unskilled  personnel  are  discussed.  The  only 
decisions  required  of  the  operator  are  minor  adjustments,  test  initiation, 
and  the  like.  The  checkout  is  controlled  by  a  master  tape,  so  that  tests 
can  be  changed  by  merely  changing  the  program  tape.  About  five  tests 
per  second  are  to  be  accomplished. 

One  kind  of  problem  was  that  in  which  the  engineering  solution  was 
reasonably  straightforward,  but  the  implementation  would  add  enormously 
to  over-all  system  complexity.  An  illustration  is  given  for  the  case 
where  seven  bits  of  information  are  sensed  to  energize  2'  or  128  relays. 

A  large  number  of  gates  and  flip-flops  may  be  necessary  to  ensure  this 
(logically)  simple  capability. 

A  second  problem  area  wa6  that  of  "non -off-the-shelf"  items.  If  one 
needs  a  counter  which  is  to  be  controlled  by  binary  code,  it  may  be 
necessary  to  modify  extensively  a  conventional  counter.  Or  the  designer 
may  have  to  desigi  his  own  special  meters  to  meet  peculiar  measurement 
needs. 

Sometimes  the  physical  realities  are  in  direct  conflict.  For  example, 
it  may  be  desirable  to  obtain  a  filtered  voltage  reading  within  a  very 
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short  time  Interval.  But  adequate  filtering  takes  time,  and  something 
must  be  compromised. 

Despite  such  problems,  ACE  capabilities  are  improving  rspidly.  The 
author  predicts  a  further  trend  toward  high-speed,  automatic  digital 
test  devices;  he  states  thet  the  most  important  step  in  a  practical 
ACE  program  is  the  detailed  statement  of  test  requirements.  These 
should  be  instrumented  first,  before  luxury  features  are  added. 


26.  Willow  Run  Laboratories,  Proceedings  of  the  Symposium  on  Digital 

Simulated  Techniques  for  Predicting  Performance  of  Large  Scale  Systems, 
Report  No  .  235^-33-x,  May  23-25,  I960'. 

A  collection  of  about  twenty  papers  pertaining  to  the  digital  simulation 
of  large  systems .  Generally,  these  systems  are  military  defense  or 
detection  systems,  not  computers.  The  digital  techniques  discussed  are 
specialized  and  do  not  involve  the  use  of  a  general-purpose  computer. 
Maintenance  and  checkout  are  not  considered  in  detail. 
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III.  MAINTAINABILITY  DIRECTIVES  AND  GUIDES 
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27-  Air  Force  Ballistic  Missile  Division,  Design  for  Maintainability  Program 
for  Weapon  and  Space  Systems,  AFBM  Exhibit  59-32,  June  l6,  I960. 

"The  overall  objective  of  this  program  is  to  obtain  optimum  ease  and 
econony  in  all  maintenance  required  in  support  of  a  weapon  and/or  space 
system.  To  achieve  this  objective,  it  is  a  fundamental  requirement 
that  maintainability  must  be  considered  in  the  initial  stages  of  design 
and  engineering.  Maintenance  tasks  which  are  required  must  be  identified, 
and  provision  made  for  these  actions.  Maintainability  must  be  engineered, 
designed,  and  built  into  the  operational  hardware,  support  and  ancillary 
equipment,  and  check-out  and  maintenance  components  of  the  weapon  or 
space  system." 

To  accomplish  the  above,  the  contractor  is  required  to  prepare  an 
explicit  maintainability  program  for  his  weapon  system,  identify  the 
organization  to  implement  this  program,  have  this  organization  provide 
design  engineering  with  maintainability  guidelines,  include  pertinent 
maintainability  criteria  in  each  equipment  design  specification,  and 
provide  for  functional  and  environmental  testing  of  the  desigi  to 
demonstrate  that  maintainability  requirements  have  been  met. 


20.  Department  of  the  Air  Fbrce,  Maintainability  Requirements  for  Weapon, 
Support,  and  Command  and  Control  Systems,  and  Equipment,  MTL-M-26512A, 
December  5>  19&0- 

This  specification  "establishes  the  maintainability  design,  analysis, 
and  data  requirement s  for  weapon  systems,  support  systems,  and  command 
and  control  systems,  and  Air  Fbrce  equipment  and  provides  policy  and 
general  procedures  for  a  maintainability  program  by  the  contractor." 

From  the  initial  design  stages,  the  contractor  is  required  to  include 
maintainability  as  a  specific  area  of  effort,  identify  it  as  a  part  of 
the  over-all  system  analysis,  and  predict  maintainability  characteristics. 
This  program  shall  be  carried  forward  by  a  "single  organization  or  group. 
The  contractor  shall  identify  this  single  organization  or  group,  and 
shall  define  clearly  its  responsibilities.  The  contractor  shall  also 
stipulate  the  authority  delegated  to  this  organization  to  enforce  its 
policies  and  to  ensure  necessary  action  toward  maintainability." 

"Proposals  for  new  systems  shall  include  quantitative  maintainability 
objectives"  and  if  "maintainability  parameters  are  not  known, . . . (the 
contractor)  shall  estimate  the  maximum  permissible  values  of  the 
maintainability  parameters."  Care  must  be  taken  that  "criteria, 
pertinent  to  the  design  for  maintainability  of  the  manufactured  product, 
are  provided  for  the  use  of  each  design  engineer,  and  the  data  provided 
...shall  be  updated  on  a  continuing  basis.” 

General  maintainability  principles  are  listed  and  some  design  trade-offs 
involved  in  operational  effectiveness  or  logistics  mentioned. 
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Three  definitions  of  maintainability  are  presented: 

"a.  Maintainability  (Organizational).  -  The  capability  of 
an  equipment  to  be  returned  to  an  operational  status 
in  a  specified  period  of  time. 

b.  Maintainability  (Field).  -  The  capability  of  an  equipment 
to  be  returned  to  a  serviceable  status  with  specified 
test  and  repair  equipment  within  a  specified  period  of 
time. 

c.  Maintainability  (Depot).  -  The  capability  of  an  equipment 
to  be  overhauled  and  returned  to  a  serviceable  condition 
at  a  specified  per  cent  of  unit  cost. 

"Availability  is  the  ratio  between  the  time  the  article  is  available 
for  use  and  the  total  time  (time  available  for  use  plus  time  not 
available  because  of  maintenance  being  done  or  repairs  being  made)." 


29.  Dunlap  and  Associates,  Inc.,  Human  Factors  Design  Standards  for  the 

Fleet  Ballistic  Missile  Weapon  System,  Vol.  1,  Bureau  of  Naval  Weapons, 
OD  18413,  June,  i960,  chap.  7 • 

A  method  for  scoring  the  maintainability  of  an  equipment  is  presented. 

A  large  number  of  guidelines  cure  listed,  and  each  has  been  pre-rated 
according  to  four  factors: 

1.  Importance,  i.e. ,  the  extent  to  which  down  time  is  reduced 

2.  Effect  on  available  equipment  volume 

3.  Effect  on  equipment  cost 

k.  Effect  on  equipment  reliability 

Ratings  are  based  on  whether  the  guideline  is  believed  to  have  a  large, 
medium,  small,  or  no  effect  on  each  of  the  four  factors.  For  example, 
one  guideline  might  state  that  a  power  on-off  light  should  be  provided. 
In  a  given  application,  this  might  be  Judged  quite  important  (high 
score  on  Factor  1)  and  yet  have  no  si&iiflcant  influence  on  gear  volume, 
cost,  and  reliability  (low  score  on  Factors  2,  3>  and  4).  The  design 
implication  is  clear:  put  in  the  indicator.  Fbr  other  situations, 
the  design  implication  is  more  complex  because  high-  and  low-factor 
components  must  be  traded  off. 


30.  Fitzpatrick,  Robert,  et  al. ,  The  Design  of  Test  Devices  for  Preventive 
Maintenance  of  Ground  Electronic  Equipment,  RADC  Technical  Report  58-172, 
September,  1958,  77" PP- 

The  objective  of  this  study  was  to  develop  test  measurement  techniques 
(from  the  human  engineering  point  of  view)  for  the  preventive  mainte¬ 
nance  of  ground  electronics  equipment  by  relatively  inexperienced  OS  3- 
and  5-level  maintenance  men  in  the  Airways  and  Air  Comunl cations  Service. 
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An  initial  survey  indicated  that  test  measurement  requirements  in  five 
years  are  likely  to  differ  little  from  those  of  today.  Hence,  test 
measurement  requirements  were  abstracted  from  AACS  Preventive  Maintenance 
Instructions  for  a  representative  group  of  prime  equipments.  Various 
tables  were  drawn  up  to  show  the  ranges  of  electrical  properties  required. 

On  the  basis  of  the  test  measurement  requirements  and  a  series  of  ten 
design  criteria,  displays  and  controls  for  thirteen  proposed  devices 
were  designed.  Drawings  and  descriptions  of  these  devices  were  prepared. 

The  study  Indicated  that  the  proposed  devices  would  be  adequate,  and 
that  the  methods  developed  in  the  study  could  be  used  to  develop  an 
integrated  group  of  test  devices  that  would  be  widely  applicable  in  the 
Air  Jtorce. 


31.  Jblley,  J.  D.,  Jr.,  and  J.  W.  Altman,  Guide  to  Design  of  Electronic 

Equipment  for  Maintainability,  WADC  Technical  Report  56-218,  April,  1958> 

lfVpp- 

"This  guide  was  written  to  provide  principles  to  be  applied  during 
electronic  equipment  development  for  'building  in'  provisions  for  more 
effective  maintenance."  It  includes  recommendations  for  designing 
maintainability  into  equipment  and  into  the  primary  maintenance  supports-- 
maii.  nance  procedures  and  job  instructions — and  describes  a  schedule 
for  incorporating  maintainability  into  the  operating  situation.  The 
guide  is  written  for  "designers,"  including  those  who  prepare  the  military 
specifications  and  later  more  detailed  specifications,  equipment  and 
maintenance  support  developers,  monitors,  and  those  who  evaluate  the 
operational  suitability  of  systems. 

Maintainability  is  defined  as  the  rapidity  and  ease  of  performing 
preventive  and  corrective  maintenance  operations .  Reliability  is 
defined  as  the  probability  that  a  product  will  perform  suitably  over  a 
given  period  of  time,  assuming  the  product  is  used  for  its  Intended 
purpose.  Thus,  poor  maintainability  can  reduce  reliability. 

The  need  for  systematic  consideration  of  maintainability  supports  (test 
equipment,  logistics,  personnel,  tools,  procedures,  etc.,  as  well  as 
the  prime  equipment  itself)  is  stressed.  The  importance  of  coordinating 
maintainability  features  by  conferences,  periodic  reports,  and  face-to- 
face  meetings  of  all  designers  is  also  emphasized. 

Recomnended  characteristics  for  maintainable  equipment  cure  presented  in 
90  pages  of  detailed  written  and  graphic  items  for  equipment  units.  The 
coverage  includes  racks  and  cases,  interconnecting  wires  and  cables, 
accesses,  test  points,  controls  and  displays,  control-display  relation¬ 
ships,  installation  of  prime  equipment,  test  equipment,  bench  mock-ups, 
and  tools.  The  treatment  of  test  equipment  is  relatively  brief,  as 
reference  is  made  to  the  Spector,  Swain,  and  Meister  report  (see  Item  7k) . 
It  is  noted  that  the  main  advantage  of  automatic  test  equipment  is  that 
it  can  make  a  rapid  sequence  of  checks  with  negligible  variability  or 
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without  omitting  any  steps.  ACE  has  several  disadvantages:  high  cost, 
size,  weight,  and  maintenance  requirements ;  low  versatility;  need  for 
self -checking  features  which  add  to  cost  and  to  problems  of  maintenance 
of  test  equipment;  and  the  greater  likelihood  of  requiring  modification. 

Twenty-six  pages  are  devoted  to  "considerations  in  preparing  procedures." 
"Poorly  designed  procedures  can  offset  the  benefits  of  otherwise  good 
design  for  maintainability, "  and  "delaying  preparation  of  procedures 
until  after  the  equipment  is  designed  is  likely  to  yield  poor  maintaina¬ 
bility."  The  importance  of  maintenance  procedures  for  the  user  rather 
than  for  the  design  engineer  is  stressed. 

The  use  of  special  maintenance  diagrams  for  analysis  of  symptom  patterns 
is  urged.  Since  the  number  of  possible  patterns  is  very  large,  a  "pattern- 
group"  method  is  suggested.  Samples  of  maintenance  procedures  for  routine 
checks  and  troubleshooting  are  presented  with  emphasis  on  flow  diagrams 
which  relate  signal  information  to  detailed  step-by-step  instructions. 

The  intent  is  to  reduce  the  amount  of  general  electronics  information 
the  maintenance  technician  must  have  by  providing  Job  aids  that  reduce 
the  number  and  complexity  of  decisions  required. 


32.  Hall,  N.  B.,  C.  E.  Van  Albert,  and  J.  H.  Ely,  Recommended  Equipment 
Coding  to  Facilitate  Maintenance,  RADC  Technical  Report  59-^5 9, 

September,  1959,  23  pp- 

A  number  of  AFSC  GS  3-  and  5-level  maintenance  personnel  at  five  sites 
of  the  Airways  and  Air  Communication  Service  were  observed  and  interviewed 
at  preventive  maintenance  checks  and  troubleshooting  on  radar  and  electronic 
communication  equipment.  It  was  recommended  that  the  following  additional 
types  of  information  be  coded  on  prime  ground  electronic  equipment : 

"1.  Designation  of  functional  groups:  Such  a  designation  assists 
the  technician  in  thinking  of  functional  groupings  and  in 
isolating  malfunctions  to  such  groups.  The  construction  of 
much  equipment  is  such  that  to  a  large  extent  components 
within  the  same  functional  group  are  placed  together;  however, 
no  indication  is  made  on  the  equipment  of  this  grouping. 

Schematics  often  have  functional  groups  outlined,  which 
necessitates  continual  cross-reference  between  the  schematic 
and  the  equipment  proper.  This  procedure  is  time-consuming 
and,  in  some  cases,  inconvenient  when  the  technician  must 
handle  manuals  within  a  confined  space. 

2.  Indication  of  signal  path:  To  some  extent  signal  paths  are 
indicated  on  schematics  and  on  equipment,  but  in  the  latter 
situation,  they  are  usually  difficult  to  follow.  An  indication 
of  signal  path  shows  the  technician  which  functional  groups 
interact  as  well  as  the  normal  flow  of  signals  through  the 
system. 
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3.  Identification  of  test  points:  Although  test  points  are 
usually  easy  to  locate,  their  importance  or  the  sequence  in 
which  they  should  be  used  is  seldom  indicated  on  the  equipment. 
Proper  coding  of  these  points  provides  a  suggested  test 
procedure  which  may  be  followed  by  the  technician  at  his 
option.  It  aids  in  bridging  the  gap  between  the  clean,  orderly 
circuit  diagram  and  the  less  than  orderly  equipment . 

4.  Presentation  of  historical  data  showing  obtained  values  at 
particular  test  points:  The  field  observations  and  question¬ 
naire  results  indicate  that  the  divergence  of  measured  values 
at  test  points  from  those  recommended  in  Technical  Orders  and 
Preventive  Maintenance  Instructions  confuses  technicians  and 
makes  troubleshooting  more  difficult.  It  does  not  appear 
feasible  to  change  the  values  in  the  manuals  because  the 
proper  values  appear  to  vary  from  equipment  to  equipment  and 
within  the  same  piece  of  equipment  from  time  to  time.  However, 
the  past  history  of  readings  at  test  points  taken  when  the 
equipment  was  operating  satisfactorily  can  provide  the 
technician  with  a  dynamic  standard  for  comparison  with  any 
readings  he  obtains  while  troubleshooting  the  same  piece  of 
equipment . 

These  recommendations  were  tested  by  selecting  two  identical  DuMont 
oscilloscopes,  Type  274 A,  and  providing  one  with  the  coded  information 
in  the  form  of  colored  areas  and  components,  special  patterns,  arrows 
and  other  lines  of  various  sizes  and  forms,  special  symbols,  and  arable 
numerals.  Eighteen  technicians  were  used  as  subjects,  eight  for  the 
coded  and  10  for  the  uncoded.  Average  experience  for  the  coded  group 
was  1.9  years  and  for  the  uncoded  group  3-3  years.  Twelve  malfunctions 
ranging  from  easy  to  difficult  were  introduced  and  the  times  required 
to  locate  each  were  obtained.  The  mean  (arithmetic)  time  to  find  a 
malfunction  on  the  coded  scope  was  23  minutes  (with  a  range  from  5  to 
68  minutes),  whereas  the  mean  time  for  the  uncoded  scope  was  42  minutes 
(with  a  range  from  3  to  168  minutes).  The  differences  were  statistically 
significant  beyond  the  1  per  cent  level.  The  results  also  showed  that 
coding  the  equipment  did  not  help  experienced  technicians  or  the 
identification  of  easy  malfunctions,  but  it  considerably  helped  the 
Inexperienced  technicians,  particularly  in  their  locating  and  identifying 
difficult  malfunctions. 

The  findings  and  recommendations  of  this  study  (including  examples  of 
the  coding  recoomendations)  were  presented  to  ten  RADC  engineers. 

The  first  three  recoomendations  were  accepted  by  the  engineers  with 
little  or  no  reservations,  and  the  last  with  some  reservations. 


33-  McKendry,  J.  M. ,  et  al. ,  Maintainability  Handbook  for  Electronic 

Equipment  Design,  Naval  Training  Device  Center,  Technical  Report  330-1-4, 
April,  I960,  387  pp. 

Eleven  chapters  are  devoted  to  topics  such  as  the  working  environment, 
test  equipment,  circuit  specifications,  components,  connections  and 
bousing,  function  controls,  and  labeling. 
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34.  Miller,  R.  B.,  and  J.  D.  Fblley,  Recommendations  on  Designing  Electronics 
Equipment  for  the  Job  of  Maintenance,  Air  Training  Conanand,  HRRC  Research 
Bulletin  51-33>  December,  1951  >  IS  pp. 

"It  is  important  to  know  the  maintenance  Job  requirements  in  advance  of 
production  models  of  newly  designed  equipment  in  order  that  suitably 
trained  mechanics  will  be  available  when  the  equipment  first  gets  into 
the  field. .. .Maintenance  Job  requirements  can  be  anticipated  in  three 
ways. ..  .First,  maintenance  tasks  should  be  simplified  by  the  way  the 
equipment  is  designed. .. .A  second  approach. . .involves  compiling,  under 
the  categories  of  checking,  adjusting,  troubleshooting,  replacing,  and 
repairing,  those  discriminations,  decisions,  and  actions  as  are  indicated 
by  the  design  of  the  production  model  itself.... A  third  approach  consists 
of  keeping  appropriate  records  of  maintenance  operations  which  are 
performed  during  prototype  testing  of  newly  designed  equipment.... 
Standard  Maintenance  Form  data  should  be  used  to  supplement  and  weight 
the  Job  behaviors  obtained  by  the  second  approach. 

"It  can  be  expected  that  the  systematic  integration  of  equipment  design, 
maintenance  training,  and  maintenance  operations  will  lead  to  a  higher 
degree  of  reliability  and  efficiency  in  the  use  of  the  equipment  and  a 
reduction  of  personnel  requirements  necessary  for  achieving  satisfactory 
maintenance  of  equipment." 


35.  Van  Albert,  C.  E.,  N.  B.  Hall,  and  J.  H.  Ely,  Coding  of  Equipment  for 
Ease  of  Maintenance,  RADC  Technical  Report  59-160,  October,  1959,  $  PP • 

This  report  was  prepared  for  design  engineers  and  sumnarlzes  the 
recommendations  resulting  from  a  human  factors  study  of  maintenance. 

A  description  of  the  entire  project  may  be  found  in  RADC  TR-59-159 
(see  Item  32). 
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ADDITIONAL  REFERENCES 


Wo  hi,  J.  G.,  "Dependability  of  Military  Equipment:  A  Systems  Approach, " 
Electrical  Manufacturing,  Vol.  3>  No.  3>  March,  1959>  PP*  96-100. 

Wo  hi,  J.  G.,  "Why  Design  for  Maintainability,"  IRE  Convention  Record, 
March,  1961. 
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36,  Ashar,  D.  G.,  Probabilistic  Model  of  System  Operation  With  a  Varying 
Degree  of  Spares  and  Service  Facilities/  IBM  Data  Systems  Division 
( n  •  cl  ( )  • 

"The  techniques  for  evaluating  the  probabilistic  properties  of  systems 
having  identical  operational  and  spare  units  with  different  facilities 
for  repairs  are  discussed  in  this  paper.  The  important  properties 
studied  are: 

(a)  Reliability  and  transient  behavior  of  systems. 

(b)  The  expected  values  of  times  to  failures  and  the  measure 
of  variations  from  the  distributions  of  system  failures. 

(c)  Asymptotic  or  steady  state  probability  of  such  a  system 
being  in  unavailable  state. 

"The  general  expressions  are  obtained  for  many  cases,  while  the  methods 
are  illustrated  by  a  simple  practical  example . " 

37  •  Bovalrdj  R.  L.,  and  H.  W.  Karr,  Applicability  of  Quantitative  Analysis 
to  Problems  in  Maintenance  Management,  General  Electric  (Jo.,  R  5HWP-18, 
June,  i£«,  3k  pp. 

"This  is  a  report  on  a  brief  study  to  determine  the  possibility  of 
applying  mathematics  and  economic  theory  to  maintenance  management 
problems.  In  Section  I  routine  operating  decisions  are  described  and 
analyzed  with  the  aim  of  discovering  where  quantitative  methods  of 
analysis  appear  applicable.  Long  run  decisions  are  also  considered  with 
the  same  aim. 

"In  Section  II  a  cost  minimizing  maintenance  model  is  described  which 
takes  account  of  such  factors  as  deterioration  and  failure  probabilities, 
man-hours  for  preventive  inspection  and  preventive  repair,  elapsed  time 
for  emergency  diagnosis  and  repair,  cost  of  setting  up  for  periodic 
maintenance,  cost  of  maintenance  men,  and  cost  of  system  downtime.  This 
model  is  demonstrated  with  a  numerical  example  illustrating  how  it  can 
be  used  to  determine  the  optimum  time  between  periodic  maintenance,  the 
optimum  number  of  maintenance  men,  and  at  which  periodic  to  inspect  each 
item." 

38.  Boviard,  R.  L.,  and  H.  I.  Zagor,  "Deriving  Operational  Availability  from 
Sub-System  Maintainability  and  Reliability,"  in  F.  L.  Ankenbrandt  (ed.). 
Electronic  Maintainability,  Vol.  3,  Reinhold,  New  York,  i960. 

A  method  for  estimating  availability  in  terms  of  operational  modes  and 
equipment  groups  is  presented.  Each  group  has  associated  with  it  certain 
reliability  (mean  down  time  per  failure)  data.  Equipment  survival  like¬ 
lihoods  can  then  be  obtained  by  simple  combinatorial  methods,  and  can  be 
specified  as  part  of  a  procurement  contract. 
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Often  some  operational  capability  exists  even  if  one  of  the  major  equip¬ 
ment  groups  fails;  if  enough  of  the  subsystems  fail,  however,  the  sys¬ 
tem  is  totally  ineffective.  By  setting  up  a  table  of  the  effectiveness 
under  the  failure  of  different  equipment  groups,  a  "marginal  effectiveness" 
measure  can  be  derived  for  each  group.  This  measure  can  be  used  to  sug¬ 
gest  maintenance  strategy.  For  instance,  if  restoration  of  equipment 
group  A  would  raise  over-all  effectiveness  from  .18  to  .37,  allocation  of 
principal  maintenance  effort  to  group  A  should  be  more  efficient. 

It  is  suggested  that  a  tactical  worth  parameter  (product  of  effectiveness 
and  reliability)  be  investigated  as  an  aid  to  tactical  decision  making. 

39.  Brunner,  L.  K.,  L.  J.  Clevenger,  and  G.  W.  Dodson,  A  Method  of  Identify¬ 
ing,  Quantifying,  and  Specifying  the  Objective  Predictors  of  Maintaina¬ 
bility.  Air  University  School  of  Logistics,  Wright- Patterson  AFB,  June, 
19507  68  pp,  and  three  appendixes. 

The  discussion  begins  with  a  critical  review  of  six  official  Air  Force 
maintainability  directives  (DOD  3232.1,  AF  Regulation  66-29,  Mil-M-26512, 
AMC  Manual  66-2,  AM3  Manual  66-5,  and  Proposed  AFBM  Exhibit  59-32) .  Four 
deficiencies  are  conmon  to  all:  (l)  evaluations  are  conducted  too  late  in 
the  development  cycle;  (2)  evaluation  criteria  are  too  general  and  too 
judgmental;  (3)  reliability  and  maintainability  are  evaluated  separately 
rather  than  jointly;  (4)  though  responsibility  for  maintainability  16 
shared  by  AMC  and  ARDC,  only  ARDC  has  authority. 

Nine  different  phases  in  the  development  and  production  cycle  are  de¬ 
fined  and  the  kinds  of  tests  and  reviews  in  each  phase  are  given.  A 
definition  of  maintainability  in  terms  of  "mean  time  to  repair"  is  given; 
this  measure  is  coordinated  with  the  "mean  time  to  failure"  conception 
of  reliability.  "Availability"  is  the  ratio  of  reliability  to  reliability- 
plus-malntalnability.  Besides  this  algebraic  tie  between  the  two  con¬ 
cepts,  the  point  is  made  that  "reliability. . .  can  be  designed  into  a  sys¬ 
tem  and  exist  in  a  pure  state  independently  of  maintainability  consid¬ 
erations,  . . .  but  maintainability  features  cannot  be  applied  to  a  design 
in  an  intelligent  manner  without  prior  consideration  of  component  re¬ 
liability." 

A  chapter  on  reliability  statistics  and  failure  curves  summarizes  many 
analytic  techniques.  As  yet  there  is  no  agreement  on  the  meaning  and 
combination  of  basic  maintenance  terms;  many  approaches  to  and  defin¬ 
itions  of  maintainability  are  quoted  to  illustrate  the  diversity  of 
opinion. 

In  computing  a  maintainability  index  for  competing  systems,  "delete  from 
consideration  those  factors  that  are  specified  in  the  contract  and  are 
constants  in  comparative  analysis  between  systems  during  source  selection. 
Since  they  are  beyond  the  control  of  the  contractor,  he  should  not  be 
given  credit  or  penalized  for  their  effect  on  the  overall  maintaina¬ 
bility  index." 
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Maintenance  task  time  and  frequency  are  estimated  for  each  sub-assembly. 
Formulas  for  computing  different  kinds  of  down  time  are  given.  An  avail¬ 
ability  parameter  is  suggested  as  the  best  mix  of  reliability  and  main¬ 
tainability: 

Availability  ■  MTBF 

MTBF  +  MTTO  +  (MTTIS) 

MTBIS 

where  MTBF  «  mean  time  between  failures 

MEER  -  mean  time  to  repair  failures 

MTBIS  *  mean  time  between  inspection  and  service 

M3TIS  -  mean  time  to  Inspect  and  service. 

40.  Cooper,  Joel,  "An  Integrated  Approach  to  System  Maintainability, "  in 
F.  L.  Ankenbrandt  (ed.),  Electronic  Maintainability,  Vol.  3,  Reinhold, 

New  York,  i960. 

Since  a  high  percentage  of  malfunctions  are  initiated  by  humans,  a 
method  for  estimating  the  "human  reliability"  of  a  system  would  be  use¬ 
ful  during  the  planning  and  development  sequence.  It  is  proposed  that 
classifying  maintenance  operations  and  collecting  data  on  time,  accuracy, 
and  variability  in  performing  them  would  be  useful.  Such  data  could  be 
applied,  for  example,  to  the  man-machine  allocation  problem:  If  human 
response  variability  for  a  set  of  operations  is  too  great,  then  that 
group  of  tasks  becomes  a  good  candidate  for  mechanization  or  procedural 
revision. 

Equipment  attributes,  maintenance  actions,  and  maintenance  level  are 
determining  features  in  system  design;  in  order  to  investigate  the  re¬ 
lations  among  elements  of  these  three  "primaries, "  a  three-dimensional 
matrix  is  suggested.  For  a  particular  maintenance  level  (e.g.,  depot) 
there  would  be  a  separate  two-dimensional  matrix  of  equipment  attribute s- 
by-actions . 

Regarding  the  relationship  between  this  method  and  the  SAIM  matrix 
(see  Item  31),  the  author  suggests  that  "it  would  be  possible  to  con¬ 
sider  the  system  at  a  general  level  under  the  SAIM  method  and  translate 
to  the  maintainability  matrix  as  proposed  here  with  no  problems  in 
transition. " 

41.  Davis,  R.  G.,  Preference  Structures  and  the  Measurement  of  Military  Worth, 
Princeton  University,  Econometric  Research  Program  ftmorandum  No.  *7, 
September,  i960,  124  pp. 

This  monograph  deals  with  the  determination  of  "military  worth  functions." 
•typically  the  analyst  begins  with  some  preference  Judgments  expressed  by 
competent  military  authority;  the  Judgments  may  be  rankings  or  ratings. 

By  making  assumptions  about  "rationality, "  axiom  structures  can  be  set  up 
to  analyze  such  data.  Several  axiom  systems  are  examined,  with  particu¬ 
lar  attention  given  to  the  Aumann-Kruskal  method  of  computing  numerical 
utility  functions. 
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42.  Depian,  Louis,  Reliability  Using  Redundancy  Concepts,  George  Washington 
University,  February  1,  1959,  18  pp. 

The  question  of  hov  imperfect  circuits  may  be  combined  to  achieve  im¬ 
proved  reliability  is  discussed.  Two  methods  are  described:  (l)  multiple 
circuits,  a  majority  of  which  is  required  to  provide  an  output;  (2) 
multiple  circuits  whose  output  is  averaged.  A  mathematical  treatment 
shows  the  averaging  technique  to  be  more  advantageous. 

43.  Grodsky,  M.  A.,  G.  W.  Levy,  and  R.  D.  Sorkin,  A  Model  of  Human  Maintenance 
Behavior  -  I;  Development  of  the  Problem  and  Evidence  for  a  Model,  The 
Martin  Company,  Engineering  Report  11133,  I960. 

Four  criteria  are  suggested  for  studying  the  advantages  of  automatic  or 
manned  modes  for  space  flight:  payload  weight,  economy  (over-all  opera¬ 
tional  costs),  utility  of  man,  and  mission  effectiveness. 

Previous  studies  indicated  that  application  of  these  criteria  Justified 
a  manned  vehicle.  "It  is  apparent  that  there  is  a  definite  per  cent 
weight  advantage  in  using  a  manned  mode  of  maintenance  as  opposed  to  an 
automatic  mode  of  maintenance .  It  should  be  noted  that  the  Increased 
weight  in  the  manned  mode  of  maintenance  also  Includes  the  weight  re¬ 
quired  to  support  men  within  the  system."  These  savings  may  be  attenuated 
in  the  actual  case  because  equivalent  or  greater  cost  expenditures  would 
be  necessitated  by  the  selection  and  training  of  personnel  to  perform 
maintenance . 


An  idealized  graph  shows  relationships  between  cost  and  degree  of  auto¬ 
mation.  "There  is  a  payoff  point  between  these  axes  where  the  combina¬ 
tion  between  manned  and  automatic  systems  gives  an  optimum  low  cost. 

This  'payoff  point  is  not  yet  known  and  we  are  paying  a  large  penalty 
for  this  lack  of  knowledge . . . .Our  main  Justification  for  man  within  the 
system  was  that  men  would  cost  no  more  them  an  automatic  system  and  we 
could  use  his  other  abilities  which  are  more  difficult  to  quantify,  such 
as  decision-making,  to  make  the  system  more  effective." 


Initial  steps  toward  the  development  of  a  theoretical  model  of  human  main¬ 
tenance  behavior  sure  described;  the  principal  concepts  are  taken  from 
signal  detection  theory  and  game  theory. 


44.  Lowder,  R.  G.,  Considerations  in  Prediction  of  Maintainability,  WADC  Tech¬ 
nical  Note  59-378,  November,  1959,  7  PP* 


Defining  maintainability  as  the  "capacity  for  being  kept  in  a  state  of 
good  working  order, "  this  report  reviews  the  problem  of  predicting  the 
maintainability  of  a  system  before  the  system  goes  into  production.  Sev¬ 
eral  approaches  to  maintainability  are  listed: 


1.  BuGhips  Index: 


total  time  out  of  operation  for  maintenance 
total  time  possible  to  operate 


2.  U.  S.  Army  Signal  Corps  Index:  Summed  scores  on  a  ’Ves-no" 

type  of  maintainability  checklist 


I 
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3.  General  Electric  Index:  Mean  failure  rate  for  a  given  type 

of  component 

Number  of  components  of  a  given 
type 

Mean  repair  time  for  the  given  com¬ 
ponent 

4.  Republic  Aviation:  Summed  mean-time-to-fix  for  the 

various  maintenance  task6 

Each  of  these  four  approaches  is  based  on  a  sampling  scheme.  Thus  the 
Signal  Corps  approach  samples  the  human  engineering  aspects,  the  BuShips 
index  samples  over-all  availability,  and  GE  samples  individual  component 
failures.  "In  that  sense,  each  has  taken  a  sample,  but  a  limited  one 
consisting  of  one  factor  out  of  a  population  of  factors.  Presumably,  it 
would  be  preferable  to  be  more  extensive  in  selection  of  factors  affecting 
maintainability  and  less  comprehensive  in  the  treatment  of  each." 

Twelve  factors  that  may  lead  to  a  quantifiable  index  of  maintainability 
are: 


1.  Expected  life  span  of  components 

2.  Frequency  of  reconmended  inspections 

3.  Number  of  access  parts  or  openings 

U.  Skill  level  required  of  maintenance  personnel 

3.  Number  of  adjustments  recommended  during  periodic  maintenance 

6.  Number  of  test  points  for  use  during  inspections 

7.  Number  of  built-in  test  equipments 

8.  Number  of  places  where  "quick-disconnect"  fasteners  were  not 
used  when  they  could  have  been  used 

9.  Number  of  work  spaces  that  are  too  small  according  to 
recommended  standards 

10.  Number  of  special  tools  required 

11.  Number  of  special  test  equipments  needed 

12.  Number  of  components  that  cannot  be  reached  for  inspection 
and  repair  without  first  removing  another  component 

43.  Manly,  Ron,  Maintainability  -  Its  Quantitative  Specification  and  Testing, 
Thompson  Raao-Wooldridge ,  Inc.,  Report  523-8015  (n.&.J,  28  pp. 
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Before  be  can  decide  upon  a  maintainability  design,  the  planner  has 
to  have  three  kinds  of  information: 

1.  The  general  maintenance  concept,  what  is  done  at  each  level, 
time  allowable,  etc. 

2.  Data  on  the  relative  importance  of  time,  weight,  size,  cost, 
performance,  etc. 

3.  A  method  for  predicting  the  effect  of  a  provision  for  main¬ 
tainability  on  time  or  other  system  parameters. 

A  plan  for  obtaining  adequate  maintainability  is  detailed.  Allowable 
maintainability  times,  maintenance  environment,  and  broad  concepts  of 
work  assignments  at  each  echelon  are  determined  by  military  authorities; 
the  contractor  has  to  formulate  test  equipment  and  procedures  and  show 
that  these  are  adequate  to  handle  a  definite  sample  of  failures. 

The  inclusion  of  a  definite  collection  of  failures  as  a  part  of  the 
maintainability  specification  may  have  two  advantages:  (l)  it  forces 
the  designer  to  work  through  in  step-by-step  fashion  the  equipment  and 
procedure  necessities  for  fault  location  and  checkout;  (2)  a  collection 
of  ten  or  more  troubles  can  provide  a  good  estimate  of  actual  maintain¬ 
ability  times,  even  though  no  rigorous  sampling  scheme  has  been  followed 
in  establishing  the  collection.  Of  course  the  "specified"  troubles  will 
represent  the  actual  maintainability  times  better  if  the  failures  se¬ 
lected  are  those  apt  to  be  "most  frequent,  "  and  if  failures  from  each 
of  the  system  subunits  are  Included. 

An  illustrative  test  procedure  for  a  helicopter  system  is  given. 
"Half-split"  troubleshooting  is  used,  so  that  each  conclusive  check 
should  test  about  half  as  many  subunits  as  the  previous  check. 

46.  McGrath,  J.  E . ,  and  P.  G.  Nordlie,  "Theory  of  Allocation  of  Functions 
to  Maui  and  Machines, "  in  Synthesis  and  Comparison  of  System  Research 
Methods,  Human  Sciences  Research,  Inc.,  Report  H^R-RR-oO/ 1-SM, 

February,  i960. 

Since  machines  axe  no  longer  mere  extensions  of  human  motor  functions, 
the  performance  capabilities  of  man  and  machines  cure  competitive  for 
many  tasks.  Due  to  stringent  performance  requirements  of  time,  accuracy, 
and  reliability,  the  question  of  optimum  allocation  of  system  functions 
assumes  great  importance .  Moreover,  the  limited  supply  and  the  cost 
of  the  high  operator  skills  needed  in  complex  systems  have  led  to  efforts 
to  reduce  these  high  skill  requirements . 

The  basic  problem  in  function  allocation  can  be  stated  as  follows: 

"1.  Should  human  operator(s)  be  included  in  the  system? 

2.  Where,  or  in  what  functions,  should  man  be  used? 


3.  How  can  the  system  be  designed  to  capitalize  on  special  or 
unique  capabilities  of  the  human  operator? 

4.  How  can  man  best  be  utilized  to  maximize: 

a.  system  performance  capabilities? 

b.  system  reliability? 

c.  system  versatility? 

5.  How  can  man  best  be  utilized  to  minimize: 

a.  system  cost  (original  and  operating)? 

b.  system  complexity? 

c .  development  time? 

"Vo  treat  man  as  a  component  of  a  system,  we  must  be  able  to  describe  his 
properties  and  characteristics  as  a  human  operator  In  terms  comparable 
to  those  by  which  we  describe  hardware  components.  This  set  of  terms 
must  be  quantitative,  so  that  direct  comparisons  between  particular 
machine  and  human  capabilities  can  be  made.  We  are  not  yet  capable  of 
doing  this  except,  perhaps,  for  a  small  range  of  human  capabilities.  Un¬ 
til  such  cooparisens  can  be  made,  the  problem  oust  continue  to  be  re¬ 
solved  on  grounds  less  than  systematic  and  objective. 

"In  addition,  a  logic  of  allocation  decisions  must  be  devised  which  will 
balance  performance  and  cost  factors  such  that  an  optimum  solution  re¬ 
sults  from  its  application.  Such  a  logic  must  provide  a  means  for  spec¬ 
ifying  and  measuring  pertinent  factors,  and  for  combining  factors  to  pro¬ 
vide  a  unitary  criterion  for  decision. 

"Ulus,  a  two-part  approach  is  seen  as  being  required  for  a  solution  to 
the  allocation  problem.  First,  a  common,  descriptive,  quantitative  lan¬ 
guage  must  be  developed  which  can  be  applied  to  man  and  machine  capa¬ 
bilities.  Second,  a  logic  of  allocation  decisions  must  be  formulated. 

The  two  are  clearly  interdependent.  The  problem  will  receive  major  em¬ 
phasis  in  Ihaae  III  of  this  research  program." 

Urj.  Minger,  m.  R.,  and  M.  P.  Willis,  Development  of  an  Index  of  Electronic 

Maintainability,  American  Institute  for  Research,  Report  AIR-27 5-59- 
195 9.  (Also,  Instruction  Manual  and  Evaluation  Booklet  to  accompany  the 

report. ) 

The  feintalnablllty  Index  stems  from  a  collection  of  24l  features  designed 
to  Increase  maintainability.  These  features  are  arranged  under  nine 
headings: 

1.  Displays  and  controls 

2.  External  accessibility 

3.  Test  points 

h.  Cables  and  connectors 
5 .  Internal  accessibility 
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6.  Cases 

7.  Lubricants  and  tools 

8.  tfcnuals 

9.  Test  equipment 

Each  design  feature  is  scored  according  to  its  "consequences":  mainten¬ 
ance  tine,  logistics,  equipment  damage,  and  personnel  injury.  Summed 
scores  can  be  compared  against  a  pre-ordained  maintainability  standard. 

48. 


How  many  and  what  kinds  of  men  do  you  need  to  staff  an  aircraft  mainten¬ 
ance  base?  The  general  answer  is  obvious:  it  depends  on  the  number  of 
major  units,  the  complexity  of  equipment,  the  utilization  of  weapons, 
the  proportion  of  time  that  ground  and  air  alerts  are  required,  etc. 

With  so  many  variables,  some  kind  of  a  systematic  and  self-correcting 
plan  must  be  adopted  for  separating  the  significant  factors  from  the  less 
important  ones  and  for  producing  definite  manning  tables  from  the  opera¬ 
tional  demands. 

A  traditional  method  is  to  use  "flying  hours"  as  the  basic  Indicator  of 
manpower  requirements.  Air  Force  Manual  28-1  is  a  good  illustration: 
starting  from  a  flying-hour  base,  factor  formulas  yield  organizational 
and  field  man-hour  needs  and  manning  levels.  Obese  levels  are  only  start¬ 
ing  points  and  are  subject  to  local  readjustment  because  of  variation  in 
mission,  dispersal,  facilities,  etc.  Thus,  the  optimum  manning  level  re¬ 
mains  a  matter  of  individual  Judgment. 

It  is  fundamentally  wrong  to  assume  that  Aircraft  X  requires  (say)  93*5 
maintenance  man-hours  to  produce  an  hour  of  flight,  or  that  two  flight 
hours  will  take  111.0  man-hours,  or  that  reducing  the  flight  time  to  zero 
results  in  a  zero  maintenance  workload.  The  suggested  analysis  begins 
by  separating  the  maintenance  workload  into  constant  and  variable  portions. 
The  constant  workload  is  incurred  simply  with  possession  of  the  aircraft; 
the  variable  load  is  tied  to  the  amount  of  flight  activity  and  may  be  di¬ 
vided  into  straight  line  (inspections,  changes,  etc.)  and  random  (malfunc¬ 
tions,  etc.).  As  implied  by  the  name,  straight  line  factors  are  reason¬ 
ably  linear  with  the  number  of  sorties  or  flight  hours.  Random  ground  and 
air  malfunctions,  however,  probably  bear  non-linear  relationships  to  the 
operating  schedule;  e.g. ,  it  can  be  hypothesized  that  man-hour  maintenance 
for  air  malfunctions  is  a  negatively  accelerated  function  of  flight  hours. 

Starting  from  this  broad  breakdown  of  maintenance  tasks,  a  procedure  is 
described  for  estimating  manpower  requirements.  The  type,  model,  and  num¬ 
ber  of  aircraft  to  be  serviced,  and  periodic  inspection  requirements  fur¬ 
nish  data  for  estimating  constant  workload.  If  available,  historical 
data  on  time  change  requirements  can  be  Included. 


Noriega,  V.,  F.  J.  Jowdy,  and  J.  M.  Palmer,  A  Mettudolog} 
ing  Optimum  Manning  for  Base  Level  Aircraft  Maintenance/ 


for  Determln- 
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Variable  factors  are  harder  to  estimate  but  the  Maintenance  Data  Collect¬ 
ion  System  (described  in  Air  Force  Manual  66-1)  should  provide  data  that 
can  be  correlated  against  sortie  and  flight-hour  figures.  The  authors 
anticipate  high  initial  differences  among  units  for  the  variable  manning 
requirements;  after  sufficient  data  have  been  obtained,  the  average  man¬ 
power  spent  on  variable  workload  can  become  a  quality  control  baseline. 

After  one  goes  through  the  procedure  and  has  adequate  estimates  of  average 
manpower  requirements,  "augmentation"  or  safety  factors  must  be  added  to 
the  manning  levels.  This  is  due  to  the  random  characteristic  of  failures: 
an  average  workload  if  five  units  will  occasionally  rise  to  ten  units,  and 
if  provision  has  been  made  to  cover  only  the  five,  a  backlog  will  rapidly 
pile  up.  The  authors  propose  the  classification  of  weapon  system  capa¬ 
bility  into  vital.,  essential,  and  required;  this  classification  would  be 
made  at  the  highest  command  levels .  The  vital  manpower  levels  are  aug¬ 
mented  by  a  certain  safety-factor  fraction;  lower  fractions  are  employed 
for  essential  and  required  systems. 

49-  Rone  Air  Development  Center,  Maintainability  Measurement  and  Prediction 

Methods  for  Air  Force  Ground  Electronic  Equipment,  RAW  W66-22 1,  September, 

1960,281  pp. 

This  is  the  Ihase  II  Progress  Report  of  the  RCA  Maintainability  Study 
described  in  Item  30.  The  basic  correlation  model  is  extended  into  a  field 
phase,  with  maintenance  time  the  principal  criterion.  Design  parameters 
and  personnel  factors  are  clearly  related  to  the  time  criterion  and  are 
moderately  independent  of  each  other.  Design  factors  get  the  highest  weight 
in  the  repression  equation.  Simulation  and  field  maintenance  aspects  ap¬ 
pear  to  be  similar  enough  to  permit  extrapolation  from  one  situation  to 
the  other.  Part  type,  circuitry  class,  type  of  modules  used,  and  technician 
experience  are  all  related  to  maintenance  time.  The  field  support  environ¬ 
ment  produces  severe  degradation  with  a  ratio  of  about  3  to  1  between  aver¬ 
age  field  and  laboratory  times. 

Plans  and  expected  results  for  the  next  phase  are  projected.  A  scheme  for 
validating  the  technician  proficiency  test  is  given;  it  will  study  the  re¬ 
lationship  between  test  scores  and  Job-sample  troubleshooting  times  on 
selected  FST-2  malfunctions. 

50.  Rome  Air  Development  Center,  ARe search  Planfor  Developing  Methods  of 
Maintainability  Measurement  and  Prediction,  RADC-ft»-60-*,  January,  I960. 

This  is  the  Phase  I  report  of  a  program  of  identification  and  quantita¬ 
tive  treatment  of  the  principal  factors  affecting  the  maintainability  of 
Air  Force  ground  electronic  equipment.  Two  practical  goals  are  established: 
(1)  increase  availability  of  ground  electronics  equipment,  and  (2)  reduce 
operational  support  cost.  These  goals  will  be  achieved  only  by  "the  treat¬ 
ment  of  maintainability  factors  as  fundamental,  measurable  design  criteria 
that  can  be  specified  and  controlled  from  equipment  design  inception 
through  Intended  service  life . " 


For  equipment  that  Is  required  to  be  on-line  continuously,  availability 
is  defined  by  the  ratio: 

operate  time 

Operational  availability  ■  — - 

operate  time  +  down  time 

though  both  reliability  and  maintainability  will  influence  the  availabil¬ 
ity  ratio,  a  distinction  is  made  between  these  two  concepts.  Availability 
"is  not  directly  related  to  the  occurrence  of  failure,  it  is  directly  re¬ 
lated  to  the  effects  of  failure.  It  is  the  goal  of  the  maintainability 
effort  to  reduce  the  effects  of  failure . "  Maintainability  goals  should  be 
stated  principally  in  calendar  time  and  not  in  total  man-hour  expenditure. 

Seven  factors  that  affect  the  degree  of  maintainability  are: 

1.  Quality  of  design 

2.  Operational  concept 

3.  Uest  equipment 

k.  Technical  orders  and  Instructions 

5  .  Maintenance  materiel 

6.  Personnel  factors 

7.  Maintenance  organization 

Dangers  of  over-automating  the  test  and  checkout  are  recognized:  "The  de¬ 
gree  of  automatlclty  of  performance  monitoring,  checkout,  fault-location, 
marginal  checking  circuitry,  etc.,  is  almost  unlimited;  however,  the  resul¬ 
tant  requirements  for  supporting  circuitry  can  very  quickly  override  the 
advantages  obtained  because  of  its  own  requirement  for  maintenance  and  check¬ 
out.  Also,  there  must  be  a  lower  limit  in  the  slnplicity  of  required  main¬ 
tenance  tasks  which  will  result  in  a  very  low  motivation  of  maintenance 
personnel  as  the  maintenance  task  becomes  a  fully  automatic  routine.  We 
need  to  guard  against  being  chained  by  the  magic  of  automation  without  recog¬ 
nizing  the  basic  Issues  justifying  it,  namely,  increased  productivity,  l.e., 
more  work  done  by  less  men  and  materials." 

Research  of  the  correlation  of  maintenance  task  attributes  with  the  time 
required  to  perform  the  task  is  proposed.  Thus,  separate  scores  can  be 
recorded  for  access,  packaging,  test  points,  labeling,  and  fault  Indicator 
practices.  Parameters  such  as  training  requirements,  site  factors,  and 
maintenance  instructions  can  be  also  systematically,  if  arbitrarily,  scored. 
These  scores  can  be  considered  to  be  predictors"  of  the  actual  maintenance 
time  required.  The  maintenance  time  information  will  be  obtained  from  field 
and  laboratory  observation  of  trained  technicians  performing  the  work.  It 
is  expected  that  a  multiple-correlation  analysis  will  be  carried  out  to  de¬ 
rive  empirical  weights  for  the  various  predictors. 

51.  Shapero,  Albert,  and  Charles  Bates,  Jr.,  A  Method  for  Performing  Human 
Engineering  Analysis  of  Weapon  Systems,  WADC  Technical  Report  59-784, 
September,  1959* 


The  Systems  Analysis  and  Integration  Model  (SAIM)  attests  to  indicate 
within  a  single  framework  the  significant  elements  of  a  weapon  system. 

A  two-dimensional  square  matrix  is  used.  Three  main  classes  of  ele¬ 
ments  are  those  that  (1)  determine  the  nature  and  form  of  the  system, 

(2)  constitute  the  parts  of  the  system,  and  (3)  integrate  the  parts  of 
the  system.  Detailed  breakdowns  within  these  three  classes  are  given; 
thus  performance  requirements  are  part  of  the  system  determinants,  and 
can  be  categorized  further  into  support  performance  requirements  and 
operational  performance  requirements.  Support  could  then  be  comprised 
of  handling,  storage,  and  so  on.  Subsystem  matrices  can  be  constructed 
and  combined  into  a  total  system  context. 

The  SAIM  matrix  can  be  used  to  record  the  interactions  and  Influences 
among  system  elements.  The  insertion  of  interaction  data  is  limited 
to  "direct  connections,"  one  that  does  not  go  through  a  third  system 
element.  If  a  signal  light  goes  on  and  a  "hold"  button  is  pushed,  the 
signal  light  is  not  directly  connected  to  the  button  because  of  the 
intervening  human  element  (which  is  also  listed  on  the  matrix) . 

Second,  third,  and  higher-order  relationships  can  be  shown  on  the 
matrix  also,  but  "experience  has  shown  that  usually  one  or  two  re-entries 
are  sufficient."  As  the  design  schedule  proceeds,  the  relations  be¬ 
come  more  explicit;  "each  interaction  space  in  the  matrix  can  be  con¬ 
sidered  to  be  a  file  folder  (or  a  reference  to  a  file  folder)  within 
which  is  located  the  information  pertaining  to  the  direct  relationship 
between  two  elements."  Decreasing  degrees  of  freedom  will  be  observed 
as  alternatives  are  eliminated  and  designs  are  firmed. 

52.  Terry,  Herbert,  "Game  Theory  in  Product  and  Project  Evaluation,"  paper 
read  at  Operations  Research  Society  of  America,  New  York,  May,  i960. 

The  game  theory  viewpoint  is  applied  to  several  problems  in  evaluation 
and  marketing.  For  example,  ratings  of  product  goodness  can  be  con¬ 
sidered  as  "pay-offs, "  and  item-criterion  score  matrices  can  be  ex¬ 
amined  for  "dominance"  and  the  "value"  of  alternative  strategies. 

53.  Thomas,  R.  E.,  et  al ■ ,  The  Effect  of  Various  Levels  of  Automation  on 

Human  Operator's  Performance  in  Man-Machine  Systems,  Bat telle  Memorial 
Institute,  June,  i960.  ™"~" 

If  a  system  is  under-automated,  the  operators  probably  will  be  numerous, 
overloaded,  and  "connected"  inefficiently.  If  a  system  is  over-automated, 
the  equipment  will  be  complex  and  expensive  and  man  will  be  relegated 
to  a  monitoring  role.  In  either  case,  system  performance  will  be  de¬ 
graded.  Conceptually,  at  least,  an  optimum  level  of  automation  exists. 

The  project  described  here  attempts  to  develop  an  "automation  model" 
for  the  function  allocation  problem. 

As  a  start,  three  types  of  assessments  are  postulated:  (l)  recognize 
whether  or  not  the  system  is  within  tolerance;  (2)  select  corrective 
actions  from  a  specified  repertoire;  (3)  Judge  whether  the  corrective 


action  was  effective  In  restoring  system  performance.  These  three  kinds 
of  assessments  can  be  represented  in  an  "automation  cube;"  the  automation 
level  of  any  system  can  be  defined  by  the  location  of  a  point  within  the 
cube. 

"The  first  year's  efforts  in  this  research  program  have  produced  severed 
tangible  results:  (l)  a  qualitative  model  in  which  the  automation  prob¬ 
lem  Is  reduced  to  Its  essential  features;  (2)  a  quantitative  model  that 
Includes  two  system  parameters,  a  and  b,  and  may  be  analyzed  to  determine 
the  assessment  cycle  times,  t  ,  that  will  yield  a  desired  probability  of 
control,  p  ,  for  given  tolerance  limits,  AL,  and  specified  magnitudes  of 
the  corrective  actions;  (3)  an  experimental  apparatus  that  is  capable  of 
presenting  to  groups  of  subjects  a  wide  variety  of  control  problems  char¬ 
acterized  by  the  values  of  a,  b,  p  ,  AL,  and  cl;  and  (4)  the  results  of 
pilot  experimental  studies  that  suggest  satisfactory  agreement  between 
the  predicted  and  observed  probabilities  of  control." 

54.  Wertheimer,  F.,  and  M.  Morrison,  "A  Mathematical  Model  for  Dealing  With 
Critical  Levels  of  Reliability  Cost,"  paper  read  at  Operations  Research 
Society  of  America,  Pasadena,  California,  November,  1959* 

"An  operations  research  study  is  presented  which  develops  a  method  for 
dealing  with  the  cost  of  reliability  in  system  research  and  development. 
Although  derived  for  a  major  weapon  system,  the  model  is  applicable  to 
other  systems. 

"A  model  is  constructed  which  yields  optimum  system  reliability  as 
measured  by  total  cost  to  achieve  a  stated  mission.  A  relationship 
between  levels  of  system  reliability  and  cost  to  realize  each  level  is 
assumed.  In  addition,  the  model  accounts  for  a  relationship  which  is 
postulated  to  exist  between  system  accuracy  and  reliability.  Speci¬ 
fication  of  optimum  system  reliability  on  a  cost  basis  therefore  in¬ 
corporates  concomitant  effects  on  accuracy. " 

55.  Vfrieden,  E.  G.,  "Design  Considerations  of  Built-in  Test  Equipment,"  in 
F.  L.  Ankenbrandt  (ed.).  Electronic  Maintainability.  Vol.  5#  Reinhold, 

New  York,  i960. 

Built-in  Test  Equipment  (BITE)  is  usually  expected  to  (1)  assist  during 
ground  and  in-flight  maintenance  and  (2)  confirm  proper  system  condition 
during  operational  phases  of  the  mission.  It  is  desirable  to  estimate 
the  effectiveness  of  a  given  BITE  configuration,  so  that  trade-offs  can 
be  accomplished  against  such  parameters  as  weight  and  increased  equip¬ 
ment  complexity. 

The  evaluative  parameter  used  here  is  "confidence  level, "  which  is  "the 
probability  that  when  a  test  is  performed  on  a  portion  of  a  system  and 
a  go  indication  is  obtained,  that  portion  is  operating  within  tolerance." 
Using  experimental  modifying  terms  to  express  reliability  attributes  of 
the  system,  confidence  level  formulas  are  derived  for  several  different 
operating  situations.  It  is  possible  to  reverse  the  formulas  and  de¬ 
rive  testing  intervals  from  a  confidence  level  which  is  stated  in  advance. 
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V.  HUMAN  PERFORMANCE  OF  CHECKOUT 
AND  FAULT-ISOLATION  TASKS 


56. 
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Bond,  N.  A.,  Jr.,  "Maintaining  Space  Vehicles,"  paper  presented  at  a 
meeting  of  the  American  Rocket  Society,  Los  Angeles,  May,  i960. 

This  paper  discusses  the  behavior  of  trained  personnel  performing  main¬ 
tenance  tasks  for  manned  space  flights.  The  principal  recommendations 
are  to  (1)  provide  discreteness  in  the  modularization  of  equipment  so 
that  the  technician  can  troubleshoot  as  he  does  in  a  linear  chain  of 
elements;  and  (2)  push  failure  prediction  methods  so  that  a  large  pro¬ 
portion  of  failures  can  be  anticipated.  The  achievement  of  both  these 
recommendations  will  probably  involve  major  transformation  of  equipment 
parameters  and  packaging. 

57.  Bryan,  G.  L.,  and  D.  H.  Schuster,  An  Experimental  Comparison  of  Trouble 
Shooting  Training  Techniques,  University  of  "Southern  California,  Elec¬ 
tronics  Personnel  Research  Report  No.  50,  November,  1959. 

On  the  hypothesis  that  furnishing  specific  guidance  and  explanations 
would  provide  superior  training  in  troubleshooting.  Navy  electronic 
trainees  were  given  three  days  of  instruction  and  training  using  the 
DAS- 3  Lor  an.  Three  levels  of  guidance  (none,  partial,  and  full)  and 
two  levels  of  explanation  (none  and  verbal  rationalizations  of  correct 
checks)  were  used.  A  standard  criterion  test  in  the  pull-tab  format 
showed  that  both  guidance  and  explanation  helped  the  student.  A  sub¬ 
sequent  check  stowed  that  those  who  had  received  guidance  training  made 
higher  troubleshooting  scores  on  a  live  SPS-10  radar.  The  radar  test 
was  given  by  independent  examiners  four  weeks  after  the  experiment. 

58.  Bryan,  G.  L.,  et  al..  Electronics  Troublc-Shootiig:  A  Behavioral  Analysis, 
University  of  Southern  California,  Electronics  Personnel  "Research  Report 
No.  13,  March,  1956,  206  pp. 

"Step-by-step  protocols  from  four  data  sources  are  examined  with  the  ob¬ 
jective  of  developing  a  framework  for  behavioral  analysis  of  trouble 
shooting.  In  an  atteiqpt  to  integrate  various  facets  of  such  performan¬ 
ces,  qualitative  segments  are  identified  in  each  protocol,  and  reference 
is  made  to  their  frequency,  position,  consistency,  and  behavioral  con¬ 
sequences.  Separate  treatment  is  given  to  special  aspects  such  as  re¬ 
dundancy,  errors,  time,  and  action  rate.  Throughout  the  report,  attention 
is  directed  to  generalizability  of  the  results  to  different  kinds  of 
electronic  equipment  and  to  different  test  formats." 

59*  Bryan,  G.  L.,  et  al..  The  Role  of  Humans  in  Complex  Computer  Systems: 
Maintenance,  University  of  Southern  California,  Electronics  Personnel 
Research  Report  No.  26,  January,  1959. 

"This  investigation  was  designed  (1)  to  observe  and  describe  maintenance 
requirements  and  practices  for  digital  computer  systems  now  in  use,  (2) 
to  examine  the  methods  for  screening,  training,  and  evaluating  computer 
technicians,  and  (5)  to  obtain  up-to-date  Information  on  the  organization 
and  supervision  of  computer  maintenance  work.  Visits  were  made  to  50 
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representative  computing  centers  In  the  United  States;  on-site  Infor¬ 
mation  was  collected  by  means  of  observation,  questionnaires,  and  in¬ 
terviews. 

"Three  categories  of  maintenance  activities  are... checkout,  scheduled 
maintenance,  and  troubleshooting...  .  Checkout  includes  all  the  pro¬ 
cedures  for  testing  the  computer  and  its  auxiliary  equipment  to  make 
sure  that  the  system  is  ready  for  operational  use.  Scheduled  mainten¬ 
ance  consists  of  the  maintenance  activities  which  are  planned  in  ad- 
vance,  and  which  are  accomplished  according  to  a  more  or  less  rigid 
schedule.  Troubleshooting  is  the  process  of  identifying  the  source  of 
malfunctioning  in  the  equipment  and  then  repairing  or  replacing  the  de¬ 
fective  units  or  component s. 

"It  is  recognized  that  these  categories  of  work,  though  logically  separ¬ 
able,  may  be  under  way  simultaneously  in  a  computer  center.  Only  when 
things  are  working  well  are  the  checkout  and  scheduled  maintenance  events 
completely  foreseeable.  In  a  case  of  trouble,  checkout  and  diagnostic 
programs  may  both  be  at  work  in  the  computer,  with  preventive  mainten¬ 
ance  being  done  on  certain  units  of  the  equipment  at  the  same  time." 

Many  aspects  of  the  checkout  procedure  are  fully  automated  in  today's 
computers .  "A  systematic  run-through  must  be  made  that  will  activate 
all  (or  nearly  all)  of  the  internal  commands  under  various  conditions. 
Included  in  the  procedure  will  be  programs  to  test  tape  and  card  units, 
indicator  lights,  printers,  typewriters,  and  auxiliary  memory.  Most 
computer  installations  report  that  this  run  is  accomplished  by  the 
technicians  and  not  by  the  operators.  This  is  presumably  due  to  the  fact 
that  if  something  goes  sour  during  the  checkout,  the  technician  must  be 
called  anyhow. 

"The  checkout  procedure  itself  consists  of  Inserting  specially  designed 
programs  into  the  machine,  observing  the  results,  and  taking  action  to 
meet  prescribed  machine  criteria.  These  programs  are  run  through  many 
times,  and  their  successful  completion  is  usually  accepted  as  sufficient 
evidence  that  the  coaputer  is  opexatlng  perfectly. 

"It  seems  that  the  number  of  checkout  programs  used  by  an  installation 
does  not  have  any  direct  relation  to  its  size,  but  depends  on  the  way 
the  programs  are  constructed  and  also  upon  the  number  of  different  per¬ 
ipheral  equipments  to  be  Included  in  the  checkout.  On  the  average,  an 
Installation  has  eight  checkout  programs.  One  installation  used  25 
checkout  programs,  while  another  with  the  same  model  coaputer  usee  only 
one.  In  the  latter  case,  of  course,  many  of  the  function  tests  have 
simply  been  combined  into  a  single  routine. 

"The  average  time  required  for  a  checkout  is  one  hour.  Naturally  this 
varies  considerably  from  place  to  place.  One  computer  center  takes  only 
10  minutes  to  run  the  checkouts;  another  requires  three  hours... .  A 
practical  basis  for  this  divergence  in  philosophy  is  found  in  the  way 
the  computer  is  used.  A  coaputer  enployed  primarily  for  routine  tabu¬ 
lating  purposes  does  not  have  its  capabilities  exercised  to  any  great 
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extent.  The  need  for  strenuous  checking  in  such  a  case  is  much  less 
them  for  a  computer  running  elaborate  telemetering  problems  which  are 
programmed  to  take  every  advantage  of  the  machine's  capability." 

Concerning  the  use  of  checkout  routines,  "About  half  of  the  installations 
report  that  they  get  through  it  over  75$  of  the  time  without  trouble. 
Occasionally  the  computer  won't  run  at  all;  the  technician  can’t  even 
put  his  checkout  programs  on  it.  Principal  culprits  accounting  for  this 
total  loss  of  capability  were  power  problems,  failure  of  machine  to  read 
from  card  to  tape,  improper  action  of  memory  addressing  circuitry,  and 
(rarely)  difficulties  in  the  master  clock  timing.  Technicians  are  not 
particularly  disturbed  by  this  situation,  since  the  malfunction  is  usu¬ 
ally  isolated  to  a  major  unit  very  quickly. " 

A  no-go  checkout  result  does  not  necessarily  mean  that  the  system  cannot 
be  used;  the  system  may  be  able  to  proceed  with  certain  kinds  of  data 
processing.  "Sometimes  the  computer  runs  through  checkout  perfectly,  but 
when  the  real  program  to  be  computed  is  Inserted,  it  doesn't  work  properly. 
Ibis  happens  as  frequently  as  once  a  month  for  some  Installations,  and 
for  others  it  occurs  only  once  or  twice  a  year.  Generally  this  situation 
is  attributed  to  the  fact  that  the  machine  is  not  'stressed, '  or  extended, 
as  much  by  the  checkout  programs  as  it  is  by  the  money  run.  Also,  some 
machines  are  pattern  sensitive.  That  is,  due  to  minor  design  imperfec¬ 
tions,  certain  instruction  and  storage  sequences  which  rarely  occur  will 
cause  the  machine  to  malfunction.  It  is  for  these  reasons  that  the  tech¬ 
nicians  find  it  necessary  to  modify  their  checkout  routines  and,  on  occa¬ 
sion,  to  originate  new  checkout  loops. 

"All  of  the  computer  installations  visited  use  marginal  checking.  In 
fact,  a  majority  of  them  claim  that  they  can  margin  check  over  75$  of 
the  main  computer  circuitry. 

"An  appealing  feature  of  the  marginal  checking  concept  is  its  usefulness 
in  detecting  Intermittent  failures.  Intend. ttents  have  always  been  a 
bugaboo  to  the  maintenance  man.  Ib  take  care  of  cm  Intermittent,  he  usu¬ 
ally  has  to  wait  until  the  offending  part  falls  couple tely,  or  else  be 
has  to  accoopllsh  wholesale  replacement  of  units  and  couponents.  Insofar 
as  marginal  tests  catch  intend  ttents,  they  are  indeed  a  boon  to  the  be¬ 
leaguered  technician.  There  seems  to  be  very  little  documentation  of  the 
extent  to  which  intend  ttents  can  be  trapped  in  this  manner,  but  most  of 
the  technicians  Interviewed  were  convinced  that  statistical  studies  would 
show  considerable  reduction  of  Intend  ttents  with  a  margin  system. 

. . .There  is  considerable  agreement  that  marginal  checking  is  needed  for 
large  machines  that  use  enormous  numbers  of  vacuum  tubes.  In  the  case  of 
solid-state  computers,  however,  a  controversy  is  arising  as  to  the  neces¬ 
sity  for  marginal  checking." 

The  established  maintenance  procedure  was  often  modified  to  meet  local 
conditions  or  to  accomodate  field  experience .  "All  the  maintenance  man¬ 
agers  interviewed  had  done  some  changing  and  tinkering  with  their  check¬ 
out  procedures.  Generally,  the  changes  were  aimed  at  shortening  the  time 
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required,  simplifying  the  routines,  eliminating  useless  portions,  in¬ 
creasing  reliability,  and  arranging  for  better  print-outs  of  results." 

A  discussion  of  peripheral  equipment  points  out  that  mechanical  assemblies 
often  have  more  trouble  than  the  main  frame.  Furthermore,  a  different 
kind  of  technical  skill  and  fault  location  technique  is  required  for  this 
auxiliary  gear.  Installations  may  have  from  four  to  fifty  pieces  of  per¬ 
ipheral  gear,  each  requiring  checkout  and  servicing.  "Because  of  the 
greater  amount  of  electro-mechanical  elements  in  the  peripheral,  equipment, 
it  is  not  surprising  that  only  four  Installations  spend  more  than  30$  of 
their  routine  maintenance  time  on  the  central  computer . " 

Digital  equipment  imposes  special  requirements  on  the  technician.  "The 
extreme  precision  of  relationships  in  the  equipment,  and  their  essential, 
on-off  nature,  lead  to  a  sharper  definition  of  the  problem  to  the  tech¬ 
nician.  He  camnot  slur  over  the  Indication.  He  cannot  get  by  on  vague 
intuitions  about  circuit  relationships,  as  a  good  many  people  do  in 
troubleshooting  other  types  of  gear.  The  technician  must  structure  the 
problem  in  terms  of  functional  or  logical,  interrelationships  of  the  equip¬ 
ment  rather  than  in  terms  of  quantitative  electronic  characteristics . " 
While  troubleshooting  "is  a  task  requiring  considerable  abstract  reason¬ 
ing  ability,  since  one  must  work  within  a  complex  problem  framework  which 
has  tight  symbolic  restraints, .. .sophisticated  knowledge  of  electronic 
functioning  is  not  required  of  a  digital  troubleshooter. 

"Most  of  the  installations  claim  that  the  biggest  limitation  is  that  the 
(diagnostic)  routine  doesn't  tell  the  technician  where  to  look— it  only 
shows  the  functional  status  of  the  computer,  and  still  leaves  the  tech¬ 
nician  with  the  Job  of  locating  the  bad  part.  And  there  are  times,  such 
as  with  power  supply  failure  or  input-stage  breakdowns,  when  the  diag¬ 
nostic  cannot  be  run  through  the  computer  at  all.  Diagnostics  tend  to 
be  valuable  only  for  certain  types  of  troubles,  and  with  other  types  are 
of  no  help  at  all.  A  repeated  comment  was  that  the  technicians  don't  use 
the  diagnostics  unless  they  have  to — the  program  that  w as  in  the  machine 
when  it  broke  down  is  usually  a  sufficient  guide . " 

Other  topics  treated  in  the  report  relate  to  personnel  problems  associ¬ 
ated  with  the  selection,  training,  and  supervision  of  the  maintenance 
staff. 

60.  Dale,  H.  C.  A.,  Gambling,  Intelligence  and  the  Training  of  Electronic 
Maintenance  Personnel,  Applied  Psychology  Ufali,  Cambridge  (n.dTTI 

An  experiment  was  conducted  with  40  military  maintenance  mechanics  to 
determine  the  relationship  among  the  level  of  training,  intelligence, 
and  the  use  of  relatively  unsystematic  fault-finding  methods  in  main¬ 
tenance  of  electronic  equipment.  Eighteen  of  these  men  always  vised  one 
of  the  systematic  troubleshooting  methods  demonstrated  by  the  experimen¬ 
ter.  Twenty-two  took  a  gamble  (i.e.,  took  an  action  on  the  basis  of 
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hunches  rather  than  on  the  basis  of  a  systematic  approach)  at  least  once. 

It  was  found  that  "intelligence  determines  not  the  hunches  the  man  had, 
but  whether  he  chose  to  be  guided  by  them."  Level  of  training  had  little 
effect.  It  was  concluded  that  "the  traditional  method  of  training,  in 
which  fault-finding  is  looked  upon  as  an  exercise  in  applied  electronic 
theory,  will  have  to  be  drastically  changed,  if  for  no  other  reason  than 
that  these  men  will  not  be  capable  of  assimilating  electronic  theory." 

61.  Fitts,  P.  M.  (ed.).  Human  Engineering  for  an  Effective  Air-Navi ration 
Air  Traffic-Control  System,  Silo  {state  University  Research  Pbunaatlon, 
December ,  1950 . 

(Note:  Ibis  abstract  is  restricted  to  parts  of  the  report  dealing  with 
man-machine  function  allocation. ) 

There  are  four  alternative  man-machine  systems:  (1)  fully  automatic  con¬ 
trol,  (2)  automatic  control  with  human  monitoring,  (3)  semi-automatic 
control  supplemented  by  human  performance  of  critical  functions,  and 
(1»)  primary  control  by  human  operators  who  would  be  assisted  by  effective 
data-analysls,  data- transmission,  and  da ta-di splay  equipment.  The  second 
system  probably  would  not  work  well  because  humans  are  by  nature  poor 
monitors.  In  tasks  that  call  for  long  periods  of  inactivity,  humans 
have  an  almost  universal  tendency  to  become  inattentive  and  bored  and 
sometimes  fall  asleep.  Even  if  the  system  were  arranged  to  force  the 
attention  of  the  human  monitor  at  the  time  of  equipment  failure,  it  prob¬ 
ably  would  not  be  satisfactory.  One  premise  assumed  in  considering  this 
kind  of  system  is  that  the  human  should  be  prepared  to  take  over  critical 
functions  of  air  traffic  control  in  case  of  emergency.  But  a  man  cannot 
sake  Intelligent  decisions  in  an  emergency  unless  he  has  an  adequate  un¬ 
derstanding  of  the  traffic  picture  at  the  moment  of  the  emergency  and  for 
a  short  time  preceding  it.  Thus,  the  monitor  must  keep  alert  and  thorough¬ 
ly  informed  of  the  traffic  situation  at  all  times  in  order  that  he  can  take 
over  in  emergencies,  but  a  monitoring  system  is  one  of  the  worst  -poss¬ 
ible  kinds  of  work  situations  for  the  human  to  stay  alert  in. 

Although  "humans  are  notoriously  variable  in  their  behavior  under  conditions 
of  extreme  stress,...  there  is  some  evidence  to  suggest  that  under  lower 
load  conditions  a  human,  in  some  ways,  performs  better  than  does  a  machine.” 
Human  performance  may  be  degraded  under  stressful  conditions,  but  there  is 
usually  not  a  complete  breakdown. 

"Machines  are  by  no  means  infallible,  but  in  general  they  can  be  made  to 
carry  out  specific  functions  with  fewer  errors  than  would  be  made  by  humans. 
This  raises  the  question  of  whether  safety  should  depend  on  human  alertness 
and  decision  making  or  on  automatic  machines .  Our  answer  to  this  is  an  un¬ 
qualified  assertion  that  the  primary  responsibility  for  safety  should  not 
rest  on  humans . " 

Taking  these  facts  into  account,  "we  suggest  as  an  important  working  prin¬ 
ciple  that  checking,  verifying,  and  monitoring  equipment  be  devised  that 
will  make  it  impossible  for  any  human  in  an  aircraft  on  the  ground  to 
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violate  basic  safety  rules,  such  as  assigning  two  aircraft  to  the  same 
block  of  space.  This  is  the  reverse  of  the  commonly  expressed  idea  that 
men  should  monitor  machines.  We  are  suggesting  that  machines  should  mon¬ 
itor  humans . 

"Humans  appear  to  surpass  present-day  machines  in  respect  to  the  following: 

1.  Ability  to  detect  energy  of  certain  wave  lengths. 

2.  Ability  to  perceive  patterns  and  generalise  about  them. 

3.  Ability  to  improvise  and  use  flexible  procedures. 

4.  Ability  to  store  Information  for  long  periods  and  to  remember 

relevant  facts  at  the  appropriate  time. 

3.  Ability  to  reason  inductively. 

6.  Ability  to  exercise  Judgeent. 

"Present-day  machines  appear  to  surpass  humans  in  respect  to  the  following: 

1.  Ability  to  respond  quickly  to  control  signals,  and  to  apply  great 
force  smoothly  and  precisely. 

2.  Ability  to  perform  repetitive,  routine  tasks. 

3.  Ability  to  store  information  briefly  and  then  to  erase  it  com¬ 
pletely. 

4.  Ability  to  reason  deductively,  including  computational  ability. 

5.  Ability  to  handle  highly  complex  operations,  i.e.,  to  do  many 
different  things  at  once . " 

62.  Folley,  J.  P.,  Jr.,  A  Study  of  Training  Requirements  and  Study  Prere¬ 
quisites  for  Ordnance  Maintenance  of  Arny  fialllstlc  Missile  Digital 
Computer  Systems,  Army  Ballistic  Missile  Agency,  AIR-34-3 9-FB-21 4  Report. 
October,  1959* 

"The  most  important  outcomes  of  the  study  deal  with  problems  of  designing 
effective,  integrated  systems;  with  questions  of  training  philosophy;  with 
challenging  the  point  of  view  that  automated  troubleshooting  is  the  answer 
to  effective  maintenance;  with  suggesting  and  partially  demonstrating  the 
feasibility  of  making  common  practice  a  process  that  most  design  engineers 
would  say  is  impossible  of  achievement;  with  preparing  prior  to  the  delivery 
of  the  hardware,  troubleshooting  procedures  that  will  find  virtually  any 
malfunction  in  the  system." 
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Three  Array  missile  digital  computer  systems  (Redstone,  Sergeant,  and 
Pershing)  are  critically  examined.  Findings  and  recommendations  rre 
organized  under  four  headings:  (l)  broad-scope,  (2)  core  skills  and 
knowledges,  (3)  specific  skills  and  knowledges  for  each  system,  and  (4) 
prerequisites. 

The  three  systems  differ  markedly  with  respect  to  checkout  instrumen¬ 
tation.  "The  Sergeant  system  has  almost  completely  automatic  checkout 
and  troubleshooting  facilities.  Troubleshooting  within  modules  is  done 
at  the  depot  or  factory.  The  decision-making  part  of  troubleshooting  is 
entirely  removed  from  the  technician. .. .The  Redstone,  on  the  other  hand, 
has  no  automatic  troubleshooting  features.  The  technician  is  given  some 
maintenance  aids  in  the  form  of  the  system  tester,  the  component  tester, 
and  some  test  problems.  The  technician,  however,  must  figure  out  almost 
every  move.  He  must  decide. ..what  should  be  his  next  step  in  isolating 
a  malfunction. .. .The  Pershing  (has) .. .training  requirements  similar  to 
those  of  the  Redstone,  although  probably  not  quite  so  severe. 

"It  is  fairly  clear  that  neither  of  these  extreme  solutions  is  very  de¬ 
sirable.  In  one  case  (Redstone)  there  is  a  good  chance  that  the  system 
cannot  be  maintained  at  all.  In  the  other  (Sergeant)  the  costs  go  up,  and 
the  system  may  still  not  be  maintainable  In  the  field  because  of  logis¬ 
tics  problems.  Similar  solutions  on  future  systems  are  not  likely  to  be 
any  better." 

The  author  chooses  "integrated  system  development"  as  the  best  solution. 
"It  requires  acceptance  of  a  new  philosophy  of  system  design— the  phil¬ 
osophy  that  the  system  includes  not  only  hardware,  but  maintenance  and 
operating  procedures,  technical  manual  content  and  format,  and  training 
and  selection  requirements  for  personnel. . .that  the  maintenance  procedures 
people  and  training  people  have  an  equal  say  in  decisions  about  system 
characteristics,  along  with  the  hardware  designers."  Technical  manuals 
should  be  reference  works  to  be  used  by  men  trying  to  do  specified  jobs, 
rather  than  a  collection  of  information  with  an  unspecified  or  unclear 
purpose. 

The  author  supports  his  position  by  detailed  reference  to  field  conditions 
in  the  three  missile  systems.  An  investigator  attempted  to  solve  a 
troubleshooting  problem  on  the  computer.  The  first  step  was  to  consult 
the  equipment  manual.  Before  the  manual  could  be  used,  however,  it 
appeared  that  the  troubleshooter  would  need  certain  qualifications :  a- 
bility  to  Interpret  logical  equations  and  logical  diagrams,  a  consid¬ 
erable  facility  in  using  some  papers  published  by  the  computer  manu¬ 
facturer,  and  a  knack  for  combining  system  specifics  into  a  conception 
of  troubleshooting  strategy.  After  encountering  these  requirements,  the 
investigator  "raises  a  serious  question  whether  it  is  realistic  for  the 
army  to  attempt  to  train  personnel  to  maintain  this  system,  or  others 
with  similar  requirements,  without  providing  troubleshooting  procedures 
developed  by  experts  on  the  system." 
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63.  Glanzer,  M.  A.,  R.  E.  Corrigan,  and  R.  A.  Glaser,  Retention  and  Re¬ 
training  of  Skills  in  Cross- Trained.  Personnel  (U),  Air  Force  Special 
Weapons  Center,  'technical  Report  55-15,  August,  1955  (Secret-Restricted 
Data;  abstract  Unclassified) . 

Air  Force  personnel  trained  in  mechanical  tasks  were  given  orientation 
and  practice  on  checkout  tasks  for  electronic  devices.  During  train¬ 
ing  they  used  special  job  aids, .consisting  of  a  special  checkout  pro¬ 
cedure,  that  clearly  spelled  out  what  they  were  supposed  to  do.  It 
was  also  designed  to  be  a  learning  tool.  The  mechanics  were  able  to 
learn  how  to  carry  out  the  tasks,  and  they  appeared  to  know  why  they 
were  doing  them.  "Why"  was  defined  as  the  interrelationships  among 
the  various  sub tasks.  Thus,  to  use  an  automotive  analogy,  they  learned 
how  to  check  out  the  ignition  system  and  the  carburetion  system  indi¬ 
vidually,  and  then  the  procedural  aid  showed  them  how  the  ignition  sys¬ 
tem  affected  the  carburetion  system. 

After  a  short  training  phase,  the  mechanics  were  tested  on  their  pro¬ 
ficiency  (time  and  errors)  in  performing  these  tasks.  Their  scores  were 
equal  to  those  of  Air  Force  personnel  who  had  received  several  months 1 
of  Intensive  electronics  training  and  whose  average  educational  level 
and  electronics  aptitude  were  significantly  higher.  (The  electronics 
specialists  were  originally  selected  on  the  basis  of  a  high  electronics 
aptitude,  while  the  mechanical  specialists  were  selected  for  high  me¬ 
chanical  aptitude  but  low  electronics  aptitude.  The  mechanical  specialists 
were  alleged  to  have  a  generally  lower  average  intellectual  level  than  the 
electronics  specialists.) 

Two  secondary  effects  were  observed.  First,  the  mechanical  specialists 
demonstrated  the  classical  "Hawthorne  Effect"  in  being  singled  out  as 
experimental  subjects  and  being  allowed  to  learn  to  operate  electronics 
gear.  "Operation  of  electronics  gear"  was  regarded  as  being  on  a  high¬ 
er  social -intellectual  plane.  Second,  the  regular  electronic  specialists 
evidenced  a  strong  desire  to  obtain  the  special  job  aids  in  lieu  of  the 
conventional  manuals  they  had  been  using. 

6k.  McGrath,  J.  J. ,  A.  Harabedian,  and  D.  N.  Buckner,  Review  and  Critique  of 
the  Literature  on  Vigilance  Performance,  Human  Factors  Research,  Inc., 

ASW  technical  Report  NO.  1,  December,  1959/  100  pp. 

In  addition  to  reviewing  the  literature  on  vigilance,  this  report  des¬ 
cribes  various  psychological  theories  of  vigilance  performance  and  pre¬ 
sents  gui  annotated  bibliography  of  vigilance  studies. 

Vigilance  can  be  measured  by  signals  missed,  by  signals  erroneously  re¬ 
ported,  and  by  response  latency.  Performance  shows  a  rapid  decrement 
in  efficiency,  but  stabilizes  with  the  first  10  to  30  minutes  on  a  reg¬ 
ular  watch  and  may  show  an  end  spurt  if  the  end  of  the  watch  is  known 
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by  the  observer.  Various  environmental  conditions,  display  and  signal 
characteristics,  and  physiological  factors  can  enhance  or  reduce  vig¬ 
ilance;  Individual  differences  in  susceptibility  are  large.  There  are 
several  procedural  conditions  that  can  reduce  the  amount  of  vigilance 
degradation;  presentation  of  many  artificial  signals  when  the  real  sig¬ 
nals  occur  infrequently,  scheduling  of  frequent  rest  periods,  use  of 
multiple  observers,  giving  special  Incentives,  and  providing  Immediate 
knowledge  of  results. 

65.  McGraw-Hill  Book  Company  Technical  Writing  Division,  Design  Factors  for 
Aircraft  Electronic  Equipment,  WADC  Technical  Report  56-1U8,  December . 

155 r. -  — 

This  Is  a  general  manual  for  the  designer  of  military  airborne  equip¬ 
ment,  with  eaphasls  on  maintainability-related  Items.  Comprehensive 
sections  are  devoted  to  reliability  analysis  and  to  electronic,  mechan¬ 
ical,  and  environmental  design  factors.  The  maintenance  chapter  men¬ 
tions  the  principal  variables  in  testing,  accessibility,  and  equipment 
coding.  Coverage  Is  given  to  personnel  safety  items:  Interlocks,  high 
voltage  Jacks,  Insulated  controls,  capacitor  discharge  devices,  etc. 

66.  Miller,  R.  B.,  Common  Job  Components  and  Transfer  of  Non-Specific  Habits 
in  Electronics  tine  Maintenance.  Part  II:  Some  theory  and  Hypotheses  for 
Winsfer  of  Training  of  Non-Specific  Habits  in  Line  Maintenance  Jobs, 
AfrpTkC  Project  7769  Report,  January,  1955,  pp. 

Suppose  you  have  a  group  of  line  mechanics  for  the  Q-24  and  you  want 
them  to  learn  another  equipment  such  as  the  A3A;  what  kinds  of  transfer 
effects  can  be  expected  from  one  equipment  to  another?  Considering  sup¬ 
erficial  familiarity,  the  "Q-24  men  could  talk  a  better  electronics 
game,  and  thus  be  more  impressive  to  their  supervisors  and  instructors 
...but  in  actual  job  skills  and  performance  it  is  at  least  possible 
that  present  Q-24  and  K-2  line  mechanics  would  be  little  better  off 
than  a  group  of  controls."  In  general,  "transfer  effects  would  be  either 
negative  and  transitory  or  non-existent." 

A  theoretical  analysis  of  the  transfer  mechanisms  encountered  in  non¬ 
specific  Job  similarities  is  presented.  The  surrogate  or  Implicit 
stimulus-response  is  a  central  concept;  when  organized  into  a  pattern, 
these  Implicit  relationships  seem  to  filter  out  irrelevant  material  and 
reduce  interference  effects. 

At  least  seven  factors  could  yield  positive  transfer  from  one  line  main¬ 
tenance  Job  to  another: 

1.  Learning  what  to  search  for  in  the  work  environment,  including 
job  purposes,  physical  cues,  and  symbolic  cues 

2.  Enhanced  distinctiveness  of  essential  Job  cues 

3.  Habit  patterns  for  noting  and  remembering  Information 
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4.  Concepts  of  signal  flow 

5 .  Method  for  troubleshooting 

6.  work  methods 

7.  Use  of  checklists  as  performance  guides  and  as  supports  for 
learning  optimal  elements  of  job  behaviors 

"in  general  terms  these  factors  can  accelerate  learning  to  maintain  a 
new  equipment  Insofar  as  they  provide  directly  usable  iaqtllclt  models 
for  behavior  and  In  reducing  the  'learning  duties'  characteristic  of 
the  novices  Interaction  with  training  materials  and  procedures." 

67*  Miller,  R.  B.,  et  al. ,  Survey  of  Human  Engineering  Needs  In  Maintenance 
of  Ground  Electronics  Equipment,  RADC  (Technical  Report  $4-31,  February, 
l£>4,  ^4  pp.  and  Appendix. 

Observational ,  interview,  and  questionnaire  data  from  six  Air  Defense 
sites  were  directed  to  the  definition  of  maintenance  problems  on  ground 
electronics  equipment.  Seventeen  problem  areas  emerged:  (1)  accessibil¬ 
ity;  (2)  presentation  of  technical  Information;  (3)  test  equipment; 

(4)  working  conditions;  (5)  environmental  effects;  (6)  safety;  (7)  com¬ 
ponent  size;  (8)  power  problems;  (9)  color-coding  of  equipment;  (10) 
warning  Indicators;  (11)  cable  connectors;  (12)  panel  controls  and 
meters;  (13)  circuit-switching  arrangement;  (14)  preventive  mainten¬ 
ance;  (15)  troubleshooting;  (16)  reliability;  (17)  supply.  Specific 
Illustrations  and  case  examples  are  developed  for  each  of  the  problem 
areas,  and  lists  of  research  problems  are  given. 

68.  Miller,  R.  B.,  J.  D.  Folley,  and  P.  R.  Smith,  A  Comparison  of  Job  Re¬ 
quirements  for  Line  Maintenance  on  Two  Sets  of  Electron**  ggyjjaMfr, 

Air  Force  Personnel  and  Training  Research  Center,  Technical  Report 
54-84,  December,  1954,  17  pp. 

The  "equipment-demands"  form  of  analysis  originated  by  Miller  was 
applied  to  the  Q-24  radar  and  to  the  K-l  bomb-nav  system  to  determine 
the  communal ity  of  job  activities  for  the  two  systems.  It  appears 
that  while  many  individual  behaviors  (e.g.,  reading  a  gage,  acconplish- 
ing  a  screwdriver  adjustment)  are  practically  identical  for  the  two 
equipments,  real  differences  appear  at  the  knowledge,  judgment, 
and  procedural  levels.  This  finding  suggests  the  possibility  of  estab¬ 
lishing  broad  Inter-system  similarity  at  the  procedural  level,  so  that 
troubleshooting  and  other  complex  tasks  can  be  more  transferable  from 
one  equipment  to  another. 

69.  Miller,  R.  B.,  and  E.  B.  Slebodnick,  Research  for  Experimental  Investl- 
gation  of  Transferable  Skills  in  Electronic  Maintenance,  AFPTRC  Techni¬ 
cal  Report  58-2/  January,  i958~. 
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Two  experiments  were  conducted  on  transferable  skills  in  electronic  main¬ 
tenance.  The  first  dealt  with  specific  and  nonspecific  factors,  the  sec¬ 
ond  with  troubleshooting  principles. 

Experiment  1  used  24  male  high  school  seniors  divided  into  four  groups  on 
the  basis  of  intellectual  level.  Hie  Search  Group  was  given  pre-train¬ 
ing  on  how  to  filter  relevant  from  irrelevant  information  from  Job  in¬ 
structions  and  maintenance  diagrams.  Hie  Troubleshooting  Group  was  taught 
three  general  rules  of  troubleshooting.  Hie  History  Group,  essentially 
the  control  for  the  Troubleshooting  Group,  was  given  a  brief  introduction 
to  the  history  of  electronics.  Hie  Search  Control  Group  worked  with  the 
same  Job  instructions  and  diagrams  used  for  practise  exercises  in  the 
training  of  the  Search  Group,  but  they  were  not  taught  "search"  principles. 

Following  the  pre-training,  all  subjects  were  given  specific  training  on 
checking,  adjusting,  and  troubleshooting  a  single-chassis  Hi-Fi  televis¬ 
ion  receiver,  a  dual-chassis  TV,  and  an  audio  frequency  signal  generator. 

Hie  results  gave  positive  answers  to  the  following  six  questions: 

1.  Does  learning  what  to  search  for  in  complex  stimulus  environ¬ 
ments  aid  in  learning  and  retention  of  a  task? 

2.  Does  learning  of  general  problem-solving  principles  in  a  rela¬ 
tively  abstract  practice  environment  transfer  to  specific  task 
situations  when  the  subject  may  have  the  option  of  memorizing 
and  applying  a  set  of  routinized  prescriptions  as  an  alterna¬ 
tive  to  the  application  principle? 

3.  Will  different  measures  of  retention  have  different  effects  in 
experimental  and  control  conditions? 

4.  Do  different  non-specific  skills  affect  performance  of  specific 
tasks  differently? 

5.  Does  spontaneous  development  of  non-specific  transfer  of  train¬ 
ing  habits  occur  in  the  course  of  Job  experience,  without  formal 
training  in  such  habits? 

6.  Does  level  of  intelligence  have  bearing  on  the  generalization 
of  non-specific  habits  to  specific  situations,  and  are  there 
interaction  effects? 

70.  Nickels,  J.  B.,  "Using  the  Human  Element  to  Predict  Eq’-iipnent  Maintain¬ 
ability,"  in  F.  L.  Ankenbrandt  (ed.).  Electronic  Maintainability,  Vol.  3, 
Reinhold,  New  York,  i960. 

The  Diagnostic  Ability  Test  is  described.  In  its  original  form,  the  test 
consisted  of  17  performance  items  ("mount  the  1000-ohm  resistor  on  the 
board")  and  153  verbal  items  ("kilocycle  is  to  what  as  mi 111 ampere  is  to 
current").  By  means  of  item  discrimination  statistics,  engineering  re— 
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view,  and  correlation  of  item  performance  with  AFSC  skill  level,  the  test 
has  gone  through  several  stages  of  refinement. 

Once  the  DAT  is  available,  it  is  possible  to  get  test  scores  and  actual 
"time-to-fix-it"  scores  from  contract  engineers  and  Air  Force  personnel 
and  thus  establish  "specification  graphs"  or  ability  bands  for  a  certain 
class  of  equipment.  Such  a  band  might,  for  example,  include  95  per  cent 
of  the  people  at  the  AFSC-7  levelj  questions  regarding  the  proficiency 
overlap  between  this  group  and,  say,  the  contractor  engineers  can  then  be 
investigated  graphically.  An  extension  of  the  method  occurs  when  an  equip¬ 
ment  has  been  maintained  only  by  company  engineers;  from  performance  data 
on  them  and  a  study  of  the  relevant  specification  graphs  for  Air  Force 
people,  predictions  can  be  made  of  the  maintenance  times  to  be  expected 
if  Air  Force  personnel  had  to  do  the  job  alone. 

A  positive  correlation  has  been  observed  between  DAT  scores  and  individual 
maintenance  performance  on  the  AN/FST-2  Data  Transmitting  Set. 

71.  Peterson,  R.  0.,  and  E.  M.  Jones,  Core  Training  for  Electronic  Mainten- 
ance:  Principles  and  Techniques,  AFPTRC  Technical  Memo  57-2,  November, 

1557,-51  pp. - 

"A  job-oriented  approach  was  followed  in  deriving  core  training.  Train¬ 
ing  content  was  identified  through  an  analytic  study  of  maintenance  act¬ 
ivities.  Hie  activities  were  analyzed  to  identify  skills  and  knowledges 
actually  necessary  to  perform  them,  and  which  were  common  across  main¬ 
tenance  positions.  A  substantial  group  of  core  skills  and  knowledges  e- 
merged.  They  were  closely  related  to  the  conduct  of  maintenance,  and 
quite  different  in  character  from  the  'fundamental  technical  knowledge' 
content  of  typical  initial  maintenance  courses." 

Two  classes  of  transfer  mechanisms  are  distinguished.  The  identical  ele¬ 
ments  class  relates  to  specific  equipment  details:  which  tool  to  use, 
how  to  locate  a  part,  how  to  attach  a  meter,  etc.  Transfer  to  the  job 
situation  then  becomes  primarily  a  matter  of  remembering  these  activities 
until  the  job  situation  calls  for  them.  A  second,  more  complicated  mech¬ 
anism  might  be  called  transfer  of  common  principles,  precautions,  and 
performance  rules. 

72.  Rigney,  J.  W.,  "Improving  Trouble  Shooting  by  Training,"  in  F.  L.  Anken- 
brandt  (ed.).  Electronic  Maintainability,  Vol.  5#  Reinhold,  New  York, 

I960. 

Troubleshooting  is  not  now  emphasized  in  military  and  industrial  train¬ 
ing  schools,  for  several  reasons:  expense,  relative  ambiguity  of  the  sub¬ 
ject  matter,  and  curriculum  inertia  which  perpetuates  old  methods  of 
training.  But  suppose  that  troubleshooting  is  to  be  taught  explicity; 
what  are  the  most  promising  techniques  to  exploit?  , 

"First,  it  is  clear  that  trouble  shooting  skill  can  be  improved  by  giv¬ 
ing  technicians. . .highly  enriched  practice  in  solving  symbolic  trouble 
8 hooting  problems.  This  practice  gets  at  the  crux  of  the  trouble  shooting 
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task.  To  be  sure,  it  omits  the  actual  equipment,  the  problem  of  find¬ 
ing  test  points  in  often  confusingly  arranged  components,  and  the  very 
important  task  of  employing  test  equipment  properly.  However,  we  main¬ 
tain  that  unless  the  technician  is  able  to  trouble  shoot  efficiently 
at  a  symbolic  level,  it  would  be  of  little  avail  to  him  to  be  skilled  in 
these  other  aspects  of  trouble  shooting. 

"Secondly,  this  approach  to  trouble  shooting  is  the  most  efficient  one 
that  we  know  of  as  yet.  Hie  technician  is  given  a  great  deal  of  prac¬ 
tice,  with  continual  feedback,  in  a  relatively  short  period  of  time,  be¬ 
cause  there  need  be  no  waiting  for  repair  of  equipment,  insertion  of 
faults,  or  peaking  up  of  equipment  in  between  problems.  The  paper-and- 
pencil  trouble  shooting  problems  can  be  reproduced  en  masse,  and  can  be 
administered  to  large  groups  of  technicians  at  a  time.  This  is  not  poss¬ 
ible  with  live  equipment. 

"Hie  current  fashion  in  maintainability  seems  to  be  the  development  of 
automatic  or  semi-automatic  checkout  and  fault-locating  systems.  This, 
coupled  with  the  modularization  of  equipment,  will  go  a  long  way  toward 
providing  what  one  investigator  has  called  the  necessary  discreteness  of 
units  and  of  checking  sequences.  These  features  unquestionably  are  nec¬ 
essary,  in  view  of  the  order  of  complexity  one  finds  in  today's  systems. 
Without  them,  the  performance  of  corrective  maintenance  might  be  imposs¬ 
ible.  However,  it  would  be  an  error  to  suppose  that  trouble  shooting 
can  be  removed  completely  from  human  hands.  There  is  a  great  danger 
that  the  appealing  concepts  of  automation  will  obscure  the  critical  need 
for  improving  trouble  shooting  proficiency  of  the  technicians  in  the 
field,  yet  these  men  will  continue  to  be  responsible  for  keeping  these 
systems  on  the  air . " 

73.  Shriver,  E.  L. ,  De terming  Training  Requirements  for  Electronic  gystem 
Maintenance;  Development  and  Test  of  a  New  Method  of  Skill  and  Knowledge 
Analysis,  HUMRRO,  Technical  fteport  63,  June,  l9fr6>  lOO  PP> 

"In  the  present  study,  methods  were  developed  that  were  designed  to  de¬ 
rive,  from  a  system's  preproduction  information,  a  set  of  data  that  would 
be  sufficient  for  the  initial  establishment  of  a  complete  training  pro¬ 
gram  for  repairing  and  operating  that  system.  The  development  of  these 
methods,  their  application  to  an  existing  radar  system,  and  the  testing 
of  the  repair  proficiency  of  men  trained  in  the  content  derived  by  the 
methods  are  described  in  this  report. 

"The  two  basic  assumptions  that  guided  the  development  of  the  FORECAST 
methods  of  analysis  are: 

(1)  Any  job,  including  those  involving  logical  reasoning,  may  be 
described  in  terms  of  certain  cues  which  a  man  receives  through 
his  senses,  and  respones  which  he  makes  to  accomplish  his  job. 

(2)  Learning  the  cues  and  responses  and  the  cue-response  association 
which  describe  the  Job  equips  the  man  for  doing  his  Job  better 
than  learning  any  other  type  of  information;" 
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The  specification  of  cues  and  responses  for  the  operator  task  are  rela¬ 
tively  straightforward  because  the  equipment  (hopefully)  has  been  de¬ 
signed  so  that  required  responses  are  connected  with  distinctive  dis¬ 
play  conditions.  Task  definition  for  maintenance  is  more  intricate, 
and  a  specially  modified  cue-response  correlation  scheme  seems  to  be 
necessary.  Essentially,  the  method  proposed  divides  the  system  into 
four  conceptual  levels:  (l)  channel,  (2)  channel  segment  or  "gray  box," 
(3)  tube  chain,  and  (4)  part.  Each  level  has  its  own  cue.  Thus,  if  a 
malfunction  occurs,  symptom  cues  are  related  to  the  block  diagram, 
gray  box  waveforms  to  particular  check  points,  resistance  readings  at 
terminal  points  of  tube  chains,  and  resistance  readings  across  parts. 

An  experimental  training  program  was  devised  using  the  cue-response 
philosophy.  Tolerances  and  diagrammatic  indications  for  each  level  of 
response  were  set  up  to  facilitate  learning.  The  experimental  course 
required  12  weeks  of  training.  The  performance  of  graduates  of  this 
course  was  compeared  with  that  of  a  similar  group  which  had  taken  the 
conventional  30-week  training  sequence .  The  criterion  was  a  nine-day 
performance  test  on  portions  of  the  M33  Anti-aircraft  Fire  Control  Sys¬ 
tem.  No  significant  differences  in  proficiency  were  found  between  the 
two  groups,  with  the  conclusion  that  the  experimental  method  is  equally 
effective  in  terms  of  final  skill  and  over  twice  as  efficient  in  terms 
of  training  time . 

74.  Spector,  P.,  A.  D.  Swain,  and  D.  Meister,  Human  Factors  in  the  Design  of 
Electronics  Test  Equipment,  RADC  Technical  Report  55 -83,  April,  1955, 

121  pp. 

Electronic  maintenance  is  an  information-handling  situation  in  which 
three  conqponents  interact:  the  maintenance  man,  his  test  equipment, 
and  the  equipment  being  maintained.  Differences  in  ability  among  Air 
Force  3,  5,  and  7  level  (i.e.,  semi-skilled,  moderately  skilled,  and 
highly  skilled)  technicians  are  considered  with  relation  to  test  equip¬ 
ment  use.  Several  proposed  solutions  to  the  test  equipment  problems 
are  examined,  and  it  is  concluded  that  human  engineering  design  of  test 
equipment,  as  suggested  in  the  report,  was  the  most  adequate  solution. 
The  use  of  strictly  go/no-go  test  equipment  was  rejected,  primarily  be¬ 
cause  of  limited  maintenance  flexibility  and  ancillary  requirements  for 
a  greater  number  of  test  devices,  more  conqslex  test  equipment  circuitry, 
and  related  training  and  logistics  problems. 

The  logistics  problem  of  getting  equipment  replacements  is  partly  due 
to  non-ruggedized  test  equipment  as  well  as  lack  of  human  engineering 
design  to  minimize  damage. 

Some  of  the  inadequacies  of  auxiliary  aids,  such  as  technical  orders, 
maintenance  guides,  schematics,  and  check  lists,  are  listed,  and  human 
engineering  recomnendations  are  given  on  written  aids  for  the  technician 
to  use  with  his  test  equipment. 

The  data  in  this  study  were  collected  by  activity  analyses  and  task- 
structured  interviews  at  a  large  number  of  field  communication  and 
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control  sites.  A  ten-stage  work  cycle  vas  used  to  structure  the  observa¬ 
tions  : 

1.  Selecting  test  equipment 

2.  Locating  test  equipment 

3.  Transporting  test  equipment 

4.  Attaching  test  equipment  to  prime  equipment 

5.  Setting  up  test  equipment 

6.  Determining  test  equipment  accuracy 

7.  Identifying  test  points 

8.  Reading  test  equipment  information 

9-  Interpreting  test  equipment  information 

10.  Securing  test  equipment 

75*  Swain,  A.  D.,  Maintenance  Diagrams  for  Preventive  Maintenance  of  Ground 

Electronic  Equipment:  phase  I  Recommendations  for  Determining  Their 
Need,  foAPC  technical  Report  56-1^1, November,  1956,  170  pp. 

The  goals  of  this  project  were  "(1)  to  determine  what  maintenance  in¬ 
formation  should  be  presented  in  diagram  (graphic  aids)  form  for  preven¬ 
tive  maintenance  operations  of  ground  electronic  equipment  and  (2)  to 
decide  the  best  method  of  presenting  this  information  in  diagram  form." 

A  study  of  the  information  needed  by  the  technician  at  various  stages 
of  the  work  situation  yielded  a  baseline  for  evaluating  current  preven¬ 
tive  maintenance  diagrams.  It  was  discovered,  first,  that  much  of  the 
needed  information  was  not  found  in  current  diagrams*  second,  that  ex¬ 
traneous  or  job-irrelevant  information  was  frequently  presented  in  the 
diagrams;  and  third,  that  the  form  of  presentation  used  in  the  diagrams 
was  not  suitable  for  apprentice  mechanics.  "This  disparity  between  dia¬ 
gram  needs  and  current  diagram  design  led  to  the  conclusion  that  several 
models  or  types  of  diagrams,  called  Preventive  Maintenance  Diagrams, 
should  be  developed  specifically  for  preventive  maintenance  operations." 

76.  Swain,  A.  D.,  Maintenance  Diagrams  for  Preventive  Maintenance  of  Ground 
Electronic  Equipment:  Phase  iil-Final  Report,  RADC  Technical  Note  57-579, 
14  pp.  and  Appendix. 

This  report  summarizes  the  work  on  the  Preventive  Maintenance  Diagram 
project.  It  describes  the  16  basic  types  of  Preventive  Maintenance 
Diagrams  (PMD's)  developed,  and  the  two  scientific  reports  which  tell 
when  they  are  needed,  how  to  design  them,  and  what  potential  value 
PMD's  should  have  for  preventive  maintenance  operations. 

77.  Swain,  A.  D.,  Test  Measurement  Requirements  for  1960-62  Ground  Electronic 

Equipment;  Phase  I. _ The  Need  tor  New  Types  of  Test  Devices,  RADC 

Technical  Report  58-22,  March,  1^58,  65  pp. 

A  review  of  prime  and  test  ground  electronic  equipment  developed  in  1958 
for  use  in  the  early  1960's  revealed  that  most  existing  test  measurement 
practices  would  be  continued.  Data  on  the  use  of  test  equipment  by 
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Air  Force  Specialty  Classification  3-  and  5-level  maintenance  mechanics 
at  Airways  and  Air  Conmunications  Service  maintenance  sites  showed 
many  liabilities  in  existing  test  equipment.  Ike  following  difficulties 
are  anticipated. 

Test  devices  have  limitations  in  operability.  Examples  cited  show  com¬ 
plex  set-up  procedures,  confusing  display-control  arrangements,  error- 
likely  situations,  hazards,  dangers  of  damage,  and  requirements  for 
numerous  items  of  test  equipment  to  obtain  elements  of  prime  equipment 
information . 

Test  devices  frequently  present  information  not  immediately  couqpatible 
with  the  maintenance  action  required.  In  many  cases  the  mechanic  must 
decode,  compute,  interpret,  and  then  decide  what  to  do  rather  than  ob¬ 
tain  "command- type "  information.  Except  for  single-purpose,  or  built- 
in,  test  equipment,  there  may  not  be  an  entirely  satisfactory  solution 
to  this  problem. 

There  is  also  evidence  of  low  transferability  of  skill  in  the  operation 
of  different  test  sets  at  a  site;  even  though  the  sets  may  perform  the 
same  maintenance  function. 

78.  Swain,  A.  D.,  G.  R.  Purlfoy,  Jr.,  and  E.  B.  Slebodnick,  Maintenance 
Diagrams  for  Preventive  Maintenance  of  Ground  Electronic  Equipment: 
phase  fill.  Reconnendatlons  for  their  Design,  RADC  Technical  Note 
5?-379,  October,  I$57,  86  pp.  and  Appendixes. 

This  report  contains  (l)  recommendations  for  designing  16  basic  pre¬ 
ventive  Maintenance  Diagrams  (PMD's)  recommended  for  inclusion  in  the 
Air  Force  Preventive  Maintenance  Instructions  Technical  Order  series, 

(2)  a  description  and  analysis  of  a  mailed  questionnaire  evaluation 
by  127  maintenance  mechanics  of  22  prototype  PMD's,  (3)  statements 
suggesting  the  most  suitable  use  of  these  graphic  aids  which  were  de¬ 
signed  especially  for  preventive  maintenance  operations  on  ground 
electronic  equipment,  and  (It)  examples  of  the  16  basic  types  of  PMD's. 

A  previous  report  on  the  same  project,  RADC  TR  56-lkl,  tells  (l)  how 
to  determine  when  a  PMD  is  needed,  and  (2)  what  Information  should  be 
Included  in  the  PMD. 

The  present  report  concludes  by  predicting  that  if  future  Preventive 
Maintenance  Instructions  Technical  Orders  incorporated  FIO's,  these 
diagrams  would  be  acceptable  to  maintenance  mechanics,  and  help  reduce 
maintenance  errors,  injury  to  personnel,  damage  to  equipment,  time 
taken  to  perform  preventive  maintenance  operations,  and  the  amount  of 
on-the-job  training  required  before  an  apprentice  mechanic  can  reliably 
perform  these  operations  without  supervision.  Finally,  it  predicts 
that  acceptance  and  use  of  the  written  instructions  plus  graphic  aids 
by  all  maintenance  slteB  would  increase  the  standardization  of  preven¬ 
tive  maintenance  operations.  These  predictions  are  based  partly  on 
the  results  of  field  evaluation  of  the  PMD's  and  partly  on  a  systematic 
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analysis  of  preventive  maintenance  information  requirements  for  graphic 
aids  for  apprentice  mechanics.  This  analysis,  reported  in  RADC  TR 
56-1 41,  preceded  the  development  of  the  PMD's  and  associated  recommen¬ 
dations  for  their  design  and  use. 

In  addition  to  specific  layouts  for  the  16  types  of  PMD's,  the  report 
provides  general  design  reconmendations  on  the  following  topics: 
photographs  versus  drawings,  types  of  drawings,  symbology  and  its 
limitations,  placement  of  PMD's  in  the  written  technical,  orders,  size 
of  the  PMD's,  size  of  objects  and  printing  on  the  PMD's,  durability  of 
the  PMD's  under  typical,  field  conditions,  use  of  color,  use  of  caution 
and  warning  items,  symbols  showing  direction  of  movement,  labeling, 
call-outs,  and  call-out  lines,  and  learning  of  diagram  symbols. 

One  hundred  and  twenty-seven  maintenance  mechanics  of  all  skill  levels 
from  13  AACS  sites  examined  22  prototype  PMD's  and  furnished  question¬ 
naire  responses  regarding  them.  They  were  asked  to  assess  the  degree 
to  which  each  PMD  fulfilled  one  or  more  of  the  following  five  purposes: 
(l)  reducing  maintenance  errors,  (2)  injury  to  personnel,  (3)  damage 
to  equipment,  (It)  time  taken  to  perform  preventive  maintenance  opera¬ 
tions,  and  (5)  the  amount  of  on-the-job  training  required  before  an 
apprentice  (AF8C  3-level)  mechanic  could  reliably  perform  the  operations 
without  supervision.  The  over-all  response  was  highly  favorable.  For 
every  ten  mechanics  participating  in  the  evaluation,  about  six  replied 
favorably,  one  unfavorably,  and  three  omitted  the  question,  gave  irrel¬ 
evant  responses,  or  felt  the  PMD  in  question  was  not  applicable  to  a 
specific  purpose.  Study  of  the  irrelevant  replies  revealed  that  the 
individuals  responding  in  this  manner  did  not  comprehend  the  limitations 
and  objectives  of  the  PMD's.  This  finding  suggests  that  before  graphic 
aids  are  introduced,  the  personnel  who  will  use  them  must  be  trained 
briefly  on  their  purpose  and  application. 
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79.  Brock,  p.,  and  S.  Rock,  "Problems  in  Acceptance  Tests  of  Digital 
Computers, "  J.  Assoc.  Comput.  Mach.,  Vol.  1,  195^,  pp.  82-87. 

The  development  of  tests  to  exercise  the  computer,  and  the  determin¬ 
ation  of  the  acceptable  number  of  errors  for  quality  specifications 
are  discussed. 

80.  Brown,  D.  T.,  Error  Detecting  and  Correcting  Codes  for  Arithmetic 
Operations ,  IBM  Data  Systems  Division,  IBM  TR  00.718,  toy,  I960,  15  PP* 

Coding  schemes  of  the  A  +  B  form  are  described,  where  n  is  the 
number  to  be  conmunicatSd  and  A  and  B  are  integers.  Sufficient  con¬ 
ditions  to  achieve  single-error  detection  and  correction  are  stated 
and  several  theorems  are  proved  on  "smallest  values "  and  "residues . " 
Minimum-length  codes  for  single-error  correction  in  different  num¬ 
ber  bases  are  given  in  tabular  form. 

81.  Daggett,  N.  L.,  and  E.  S.  Rich,  Diagnostic  Programs  and  Marginal 
Checking  in  the  Whirlwind  I  Computer,  Digital  Computer  Laboratories, 
Massachusetts  Institute  of  Technology,  March  26,  1955,  9  PP* 

In  the  Whirlwind  Computer  "it  has  been  found  that  marginal  checking 
vastly  reduces  the  machine  failure  into  h  series  of  test  programs, 
each  of  which  thorougly  exercises  a  different  section  of  the  machine 
which  is  used  in  the  marginal  checking  procedure.  Marginal  check¬ 
ing  cannot  prevent  intermittent  and  total  failures  caused  by  shorts 
and  opens.  These  are  isolated  by  built-in  checking  features,  diag¬ 
nostic  programing,  signal  tracing  and  operator  experience  and  in¬ 
genuity.  These  methods  are  greatly  facilitated  by  a  special  program 
control  which  allows  a  periodically  repeated  test  program  to  be 
stopped  at  an  arbitrary  point  to  study  indicator  lights  and  signal 
wave  forms . " 

82.  Eldred,  Richard  D.,  "Test  Routines  Based  on  Symbolic  Logical  State¬ 
ments,"  J.  Assoc.  Comput.  Mach.,  Vol.  6,  No.  1,  January,  1959. 

A  philosophy  for  the  construction  of  test  routines  for  the  check¬ 
out  of  computers  is  stated:  "In  order  for  the  successful  operation 
of  a  test  routine  to  guarantee  that  a  computing  system  has  no  faulty 
components,  the  test  conditions  imposed  by  the  routine  should  be 
devised  at  the  level  of  programmed  orders.  Therefore,  it  seems  that 
the  proper  approach  to  writing  test  or  maintenance  routines  is  by  way 
of  the  logical  diagrams  of  the  system,  and  not  the  list  of  machine 
orders.  This  is  the  only  way  in  which  all  conditions  of  operation 
of  each  logical  function  can  be  uniquely  and  completely  defined  and 
all  logical  components  within  each  logical  function  can  be  made  to 
perform  the  task  to  which  they  are  assigned.  Orders  can  then  be 
programmed  to  present  all  conditions,  and  further  orders  can  be  pro¬ 
grammed  to  detect  improper  performance  of  the  logical  functions, 
thereby  producing  a  minimum  program  which  tests  and  detects  failure 
in  each  logical  component  in  the  system."  The  article  is  devoted  to 
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the  construction  of  a  routine  which  can  Itself  construct  a  test 
routine;  the  program  was  used  to  construct  a  test  routine  for  the 
Datamatic  1000.  "This  program  has  been  used  successfully  during 
the  debugging  and  production  stages  of  the  Datamatic  1000  and  is 
now  being  used  in  the  field  as  the  principal  maintenance  routine  for 
the  Central  Processor." 

83.  Electrodata  Division  of  Burroughs  Corporation,  220  Basic  System 

Preventive  Maintenance,  Field  Engineering  Standard  Practice  Instruc- 
No.  25.20.0313,  September  2,  i960  (used  with  220  Test  Routines,  Vol. 
2,  No.  A65535.) 

These  documents  are  intended  for  use  by  field  personnel  who  service 
the  220  computer .  The  maintenance  approach  is  that  "The  individual 
Diagnostic  Tests  should  be  used  as  an  aid  in  locating  machine  mal¬ 
functions,  and  as  Standard  Performance  Tests  on  the  unit  level. 

"Throughout  this  volume,  the  programming  techniques  are  as  simple 
as  possible,  consistent  with  effective,  efficient  testing.  All 
Data  Processor  tests  are  coded  in  short  iterations  to  simplify  un¬ 
derstanding  of  the  coding.  Lock  up  and  re-entry  points  have  been 
provided  to  save  trouble-shooting  time.  Simple,  logical  flow  dia¬ 
grams,  informative  operating  description,  instructions,  and  a  com¬ 
plete  table  of  errors  are  provided  for  each  test.  In  most  cases, 
the  address  of  the  error  halt  is  a  direct  index  to  the  detailed 
program  instruction  list. 

"In  keeping  with  the  philosophy  of  simplicity  of  operation,  all 
error  expression  in  this  volume  is  through  keyed  Halt  orders. 
Admittedly,  there  are  a  few  cases  where  a  printout  would  have  been 
more  desirable.  These  cases,  however,  were  few  and  far  between, 
and  would  have  involved  a  confusing  printout  iteration  which  would 
have  been  no  more  informative  than  the  Halt  order. 

"One  of  the  advantages  of  using  Halt  orders  for  error  indication  is 
that  a  handy  listing  of  possible  Error  Halts  can  be  included  with 
each  test.  Through  this  listing,  the  operator  can  Immediately  de¬ 
termine  where  in  the  test  the  failure  oocured  and  the  condition  of 
all  toggles  and  registers  are  generally  undisturbed,  in  the  majority 
of  cases,  the  address  of  the  Error  Halt  is  a  direct  index  to  the 
detailed  program  instruction  list. 

"In  all  tests,  excluding  memory  tests,  the  Error  Halt  is  followed  by 
and  unconditional  branch  to  that  portion  of  the  test  which  causes 
the  error  to  be  detected.  This  allows  the  operator  to  visually  ob¬ 
serve  the  error  occurrence.  If  the  return  pass  does  not  fail,  the 
test  will  continue  in  sequence .  Lockups  are  provided,  using  Program 
Control  Switches,  to  repetitively  re-execute  desired  sections  of  the 
tests  for  the  purpose  of  more  conclusively  isolating  a  failure. 

These  optional  lockup  features  are  specifically  outlined  in  the 
write-up  associated  with  each  test." 
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Detailed  error  explanations  are  illustrated:  '‘All  errors  are  expressed 
through  keyed  HALT  (00)  orders,  of  the  format  .OOnnOOaaaa,  where 
nn  ■  error  key,  and  aaaa  -  either  re-entry  point  after  error,  cr 
address  of  memory  location  which  failed. 

"In  error  Halts  related  to  the  memory  section,  the  contents  of  the 
'A'  or  'R  '  registers  have  significance  in  relationship  to  the  error. 

A  note  to  this  effect  will  be  found  in  the  error  table  associated  with 
the  Halts  which  have  this  feature. 

"After  an  error  has  been  detected,  and  the  Halt  executed,  the  case 
which  caused  the  failure  will  be  tried  again,  and,  if  successful, 
the  routine  will  continue  in  sequence.  If  the  repeat  is  not  success¬ 
ful,  the  Halt  will  again  be  executed.  In  the  case  of  the  memory 
section,  the  routine  continues  in  sequence,  after  a  Halt. 

"If  is  is  desired  to  lockup  on  a  failure,  Program  Control  Switch  /I 
should  be  set  at  the  time  of  the  error  Halt.  This  lockup  will  suppress 
the  error  Halt,  and.  allow  continuous  operation  on  the  failing  case. " 

84.  Graney,  E.  P.,  "Maintenance  and  Acceptance  Test  Used  in  MIDAC,"  J.  Assoc. 
Coaput.  Mhch.,  Vol.  2,  1955*  pp.  95-96. 

The  general  MIDAC  test  criteria  are: 

"1.  Severe  operating  conditions  for  the  computer  must  be  produced. 

2.  A  sufficiently  large  set  of  random  numbers  must  be  operated 
upon. 

3.  Elements  of  the  computer  must  assume  all  possible  states  and 
be  checked . " 

For  arithmetic  checks,  two  kinds  of  techniques  are  useful:  (l)  the 
repetition  of  a  "sample"  problem,  and  (2)  the  generation  of  a  pair  of 
random  numbers,  numerical  operation  on  the  numbers,  and  subsequent 
comparison  of  results. 

85.  Grems,  M.,  R.  K.  Smith,  and  W.  Stadler,  "Diagnostic  Techniques  Improve 
Reliability,"  Proceedings  of  the  Western  Joint  Computer  Conference, 
February,  1957>  PP*  172-178. 

This  paper  investigates  diagnostic  means  to  improve  computer  programs. 
"The  trend  toward  completely  planned  program-controlled  diagnosis  of 
errors  can  be  illustrated  by  considering  three  levels  of  diagnostic 
techniques  according  to  their  degree  of  automaticity: 

1.  Manual  techniques  use  program  tests  to  cause  a  machine  STOP 
in  case  of  an  error.  Minimum  planning  is  expended  in  the 
hope  that  a  STOP  will  not  occur. 
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2.  Semi-automatic  techniques  prepare  a  logical  pattern  of  program 
tests  and  record  data  before  a  machine  STOP.  Planning  is  re¬ 
quired  to  prestore  useful  information  before  the  occurrence  of 
an  error. 

3.  Automatic  techniques  require  extensive  planning,  as  part  of 

a  complete  system,  to  circumvent  a  machine  STOP.  A  programmed 
method  is  necessary  to  report  trouble  and  still  get  results." 

Manual  methods  are  also  described.  "Manual  diagnostic  techniques  are 
not  new  to  computing  practices  but  are  an  outgrowth  of  the  old  familiar 
check  list.  A  check  list  is  used  at  progressive  stages  during  hand 
confuting  to  explain  progress  and  to  ensure  that  completed  results  are 
similar  to  anticipated  results  at  each  stage.  The  designers  of  computers 
recognize  this  need  for  a  check  list,  and  include  machine  operation  codes 
for  conditional  and  unconditional  STOPS  and  TRANSFERS;  e.g.,  STOP,  HALT, 
MS  (Manually  Selective  Stop),  EJ  (Equality  Jump)  and  TRO  (Transfer  on 
Zero) .  The  early  computer  programs  would  be  unreliable  without  these 
operations  codes  and  an  acconpanying  check  list. 

"In  a  manual  diagnostic  routine  the  common  practice  for  using  conditional 
and  unconditional  STOP  and  TRANSFER  codes  i6  to  include  a  STOP  for  any 
questionable  situation.  Then,  when  the  STOP  is  executed,  this  manual 
procedure  follows: 

1.  Manually  copy  all  information  from  the  console  panel. 

2.  Dump  or  trace  that  portion  of  storage  containing  the  STOP 
instruction. 

3.  Scrutinize  carefully  (on  or  off  the  machine)  the  instructions 
and  the  console  information,  hoping  to  isolate  the  error. 

k.  Decide  on  a  corrective  measure. 

5 .  Employ  the  corrective  measure  in  the  program. " 

For  certain  complex  interpretations,  a  semi-automatic  approach  may 
be  indicated .  "Semi-automatic  diagnostic  techniques  include  not  only 
the  detection  and  isolation  of  errors,  but  also  the  recording  of 
comments  and  pertinent  data  by  the  program.  They  are  a  direct  out¬ 
growth  of  the  manual  techniques  and  a  stepping  stone  to  a  fully 
automatic  routine.  The  recording  of  comnents  by  the  trail,  when 
something  is  amiss,  assures  the  engineer  that  unusual  circumstances 
are  recognized  and  therefore  increases  confidence  in  the  reliability 
of  results.  This  ability  to  isolate  errors  automatically  and  record 
pertinent  information  is  a  big  step  forward  in  diagnostic  techniques. 

It  assures  that  spot-checking  is  taking  place  during  computing  and 
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is  essential  to  computing  and  coding  systems  where  relocation  and 
storage  assignment  are  delegated  to  the  system  itself.  Currently, 
there  are  a  number  of  successful  diagnostic  routines  using  these 
techniques  for  such  computers  as  International  Business  Machines 
Model  701,  704,  and  650;  for  Remington-Rand  Models  1103  A  and  UNTVAC; 
and  for  the  Bendix  0-15* 

"Semi-automatic  techniques  are  used  in  the  algebraic  computing  system, 
BACAIC.  When  the  BACAIC  IBM  701  program  detects  an  error,  the  pro¬ 
gram  prints  the  reason  for  the  error  and  suggests  a  corrective  measure. 
It  also  prints  a  minimum  of  information  to  pinpoint  the  actual  storage 
location  of  the  error  and  the  relative  location  of  the  computing  in 
the  problem. 

"The  preceding  example  illustrates  that  the  second  level,  semi-automatic 
diagnostic  routine  leaves  a  train  of  problem  progress.  Reliability  is 
promoted  thereby  because  rather  permanent  evidence  is  available  which 
shows  the  actual  path  of  the  calculation.  Furthermore,  intermediate 
answers  allow  hand-calculated  checks  upon  the  methods  or  the  final 
results  at  any  convenient  time  when  added  assurance  is  felt  necessary. 
However,  the  routine  in  the  example  is  termed  semi-automatic  because 
analysis  of  the  trail  and  any  corrective  actions  are  performed  at 
manual  speeds  after  a  machine  STOP.  Any  such  manual  procedure  is  sub¬ 
ject  to  fumbles,  errors,  and  costly  delays  even  for  small  programs. 

More  advanced  program-controlled  techniques  are  necessary  to  Improve 
reliability  as  problems  become  very  large  and  thereby  multiply  the 
costs  of  manual  machine  procedures.  Hie  confusing  mass  of  detail  con¬ 
cealed  in  an  integrated  computing  system  prohibits  any  person  or  group 
of  people  from  tracing  trouble  in  the  earlier  fashion  of  STOP,  copy 
control  console,  consult  check-list,  consider  locations  in  storage, 
dunp  large  portions  of  storage,  and  then  retire  gracefully  for  analysis. 
These  slow  procedures  impede  cosnunication  between  man  and  machine  and 
are  contrary  to  more  sophisticated  thinking  as  expressed  in  the  de¬ 
luge  of  literature  on  'automatic  programming'." 

The  progress  in  achieving  automatic  diagnosis  is  conveyed  by  present¬ 
ing  a  routine  that  runs  down  trouble  on  its  own;  when  the  routine 
finds  evidence  of  the  exact  trouble,  it  stops  and  displays  its  results 
in  printout  form. 

86.  Hart,  E.  D.,  "Maintainability  Design  for  the  Airlines  Aironics  Market," 
in  F.  L.  Ankenbrandt  (ed.).  Electronic  Maintainability,  Vol.  3, 

Reinhold,  New  York,  i960. 

Compared  to  the  military  service,  the  airlines  generally  pay  far  less 
for  their  original  equipment,  use  it  far  more,  and  yet  have  fewer 
maintenance  problems.  Among  the  reasons  for  this  are  the  motivation, 
experience,  and  stability  of  the  technician  force.  Hypothetical 
curves  show  a  drop  in  maintenance  costs  and  a  rise  in  maintenance 
quality  as  personnel  experience  increases. 
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Simplicity  of  procedure  Is  another  favorable  Maintenance  feature.  The 
airlines  technician  can  usually  modify  his  corrective  procedures  to 
suit  local  conditions,  and  he  can  apply  his  experience  to  preventive 
maintenance  as  veil.  Detailed  maintenance  record a  are  a  central  job 
aid,  and  file  cabinets  of  every  piece  of  equipment  are  kept  right  next 
to  the  maintenance  bench.  It  is  stated  that  one  clerk  in  one  airline 
keeps  records  on  most  of  the  electronic  equipment  in  over  100  planes 
and  publishes  monthly  work  summaries.  Contrast  this  efficient  system 
with  unwieldy  military  reporting  schemes! 

Accessibility  is  a  major  equipment  attribute  which  the  airlines  favor, 
but  a  blind  principle  of  "make  everthing  accessible"  is  not  followed. 
Certain  service  adjustments,  for  example,  are  made  inaccessible  to 
prevent  inexperienced  people  from  making  adjustments .  Modular  con¬ 
struction  of  equipments  is  seldom  Justified  at  present  because  of  the 
high  cost  of  replacement  modules;  the  technician  usually  has  to 
troubleshoot  down  to  the  failed  component. 

87.  International  Business  Machines,  Development  and  Evaluation  of  Systems 
Reliability  and  Maintenance  Techniques  Ilf),  final  Report  KHKKKffi 
January  1,  1$$$,  146  pp.  (Secret)  . 

Ibis  report  discusses  means  for  Improving  reliability  and  performance, 
rather  than  maintenance  techniques. 

88.  Johnson,  R.  D.,  Hike  Hercules  Computer  Monthly  State  and  Dynamic  Field 
Test  Procedures  and  Requirements  (if).  Bell  Telephone  laboratories, 

July  24,  19??*  28  pp.  (Confidential) . 

A  list  of  field  test  for  an  analog  computer  Installation  is  given. 

89.  Kovit,  B.,  "Data  Acquisition  Link  Makes  or  Breaks  Checkout  Designs," 
Space/Aeronautics,  Vol.  34,  Ho.  4,  October,  i960,  pp.  213-219. 

"Even  the  most  sophisticated  checkout  system  is  only  as  good  as  the 
data  it  is  given  to  work  on.”  Methods  of  assuring  good  test  data 
samples  include  frequency  counting,  which  is  simple  to  instrument  be¬ 
cause  one  has  to  count  against  a  time  base;  and  voltage  measurement, 
in  which  it  is  necessary  to  monitor  both  noise  and  voltage  level  and 
then  pass  through  a  voltage-frequency  conversion  before  making  a  go/ 
no-go  decision. 

Direct  cables  are  almost  universally  applied  in  missile  test  sequences; 
these  cables  provide  test  confidence  and  relative  freedom  from  ex¬ 
ternal  interference.  On  the  negative  side  are  cross-talk  problems, 
outside  exposure,  and  rough  treatment.  "It  was  certainly  no  wonder 
that,  on  a  missile  like  the  Martin  Matador,  over  half  of  all  abort- 
causing  failures  should  occur  in  either  the  instrumentation  or  the 
checkout  cabling."  Furthermore,  if  signal  levels  are  extremely  low, 
amplification  is  required.  It  is  speculated  that  "future  systems  will 
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eliminate  these  amplifiers  wherever  possible  and  use  digitizing  ele¬ 
ments  as  close  to  the  raw  data  source  as  possible." 

Air-link  checkout  problems  are  similar  to  those  of  telemetry.  Again, 
Immediate  digitizing  is  proposed  for  a  radio  link.  For  the  future,  a 
single  universal  test  center  could,  by  coded  air  data  links,  replace 
large  numbers  of  the  special  (but  overlapping)  test  equipments  now 
in  vise.  Such  a  master  central  scheme  could  be  closely  Interleaved 
with  the  computer  in  the  missile  itself,  since  using  the  prime  equip¬ 
ments'  own  computer  may  be  "the  coming  thing  in  missile  checkout." 

Test  logic  for  the  Minuteman  missile  is  briefly  reviewed.  Outstanding 
features  are  (l)  the  extraordinary  care  taken  to  prevent  cabling 
difficulties  and  (2)  a  "hot  storage"  of  recent  calibration  tests. 

This  storage  is  used  when  Instantaneous  calibration  data  are  tempor¬ 
arily  unavailable;  the  sequence  does  not  Just  stop,  but  proceeds  on 
the  basis  of  the  most  recent  stored  tests. 

90.  Laporte,  H.  R.,  "Human  Factors  Analysis  of  an  Automatic  Checkout  De¬ 
vice,"  paper  presented  at  the  Sixth  Annual  Army  Human  Factors  En¬ 
gineering  Conference,  Ft.  Belvoir,  October,  i960,  13  PP- 

Automatic  checkout  equipment  can  save  time,  provide  reliable  measure¬ 
ments,  reduce  skill  requirements,  perform  self-tests,  and  record  test 
results.  However,  a  human  operator  will  still  be  necessary  to  apply 
power,  load  the  program,  and  initiate  the  test  sequence.  Technicians 
will  also  perform  corrective  maintenance  on  the  prime  equipment  and 
on  ACE  Itself. 

Observation  of  a  particular  checkout  device  in  action  showed  that 
certain  operating  difficulties  existed:  e.g.,  controls  and  displays 
were  mixed,  insufficient  status  information  was  provided.  Interlocking 
facilities  were  inadequate.  Panel  redesign  eliminated,  to  a  large 
degree,  the  previous  inadequacies  and  provided  a  more  satisfactory 
checkout  capability. 

91.  McNaughton,  R.  A.,  "Automation  for  Quality  Control  Testing  of  Electron 
Tubes,"  IRE  Trans,,  Reliability  and  Quality  Control,  PGRQ3-9>  September, 

i960. 

"Prior  to  the  mld-Forties  most  electron  tubes  were  production-  and 
quality-tested  on  an  attributes  (go-no-go)  basis  only. ..saqple  sizes 
were  moderate  and  statistical  paper  work. ..a  minimum.  Since  that 
time  quality-control  testing  has  increased  by  leaps  and  bounds  under 
the  pressure  for  high  reliability,  performance,  and  uniformity... 
additions  to  the  quality-control  workload  have  resulted  in  the  addition 
of  statisticians,  clerks  and  typists  to  the  personnel  of  the  quality 
sections,  an  increase  in  the  number  of  test  sets  and  operators  and  an 
increase  in  the  number  of  life-test  sockets  to  accommodate  the  life  re¬ 
quirements...  an  appreciable  portion  of  the  tube  manufacturing  cost  is 
Involved  in  the  quality-control  testing  and  statistical  evaluation 
procedures. 
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"in  an  effort  to  discover  ways  and  means  for  curtailing  the  increasing 
cost  of  testing  and  the  time-lag  required  for  lot  release,  the  entire 
quality  control  process  was  reviewed  to  determine  where  automation 
could  produce  the  desired  results. 

"Since  many  of  the  measurements  contributing  to  the  increased  cost  and 
time-lag  are  made  on  the  vacuum-tube-bridge  test  sets,  and  these 
measurements  provide  the  data  for  the  statistical  work,  it  was  con¬ 
cluded  that  automation  of  the  bridge  measurements  and  statistical 
computations  would  provide  the  best  possibility  for  cost  reduction. 

"A  test  set  (was)  developed  which  performs  many  of  the  tests  auto¬ 
matically.  The  set  provides  digital  data  output  for  each  test  on 
each  tube  in  punched  tape,  reads  out  the  tape,  and  prints  a  copy  of 
the  punched  data.  The  punched  tape  is  then  fed  to  a  computer  which 
analyzes  the  data  for  the  entire  sample  and  prints  out  the  answers 
for  those  statistical  factors  such  as  lot  means,  standard  deviations, 
per  cent  deltas,  etc.,  required  for  the  attributes-variables  accept¬ 
ance  criteria  for  lot  acceptance.  The  test  set  is  designed  to  perform 
those  static  and  dynamic  tests  normally  performed  on  a  vacuum-tube- 
bridge  test  console.  It  has  twenty  test  positions  and  will  measure 
and  record  ten  different  characteristics  on  dual  section  tubes. 
Provisions  are  included  far  manually  feeding  test  data  from  other 
equipments  into  the  punched  tape  for  analysis  by  the  computer  as 
desired. 

"The  twenty  test  positions  provide  the  necessary  steps  for  making  the 
following  tests  on  dual-section  tubes:  filament,  plate,  screen-grid, 
control-grid,  and  heater-cathode-leakage  currents  under  dc  conditions; 
dc  or  ac  emission;  transconductance;  plate  resistance;  and  inter¬ 
mediate  and  cut-off-bias  plate  currents.  In  addition,  power  output 
and/or  ac  amplification  tests  may  be  substituted  into  the  test  cycle 
as  required  by  deletion  of  other  teste. 

"To  facilitate  data  processing,  the  tests  are  sequenced  so  that  those 
having  values  normally  distributed  occur  at  fixed  steps  in  the  test 
cycle.  Any  test  may  be  omitted  from  the  test  sequence  by  wiring  in 
a  skip  function  on  the  program  board.  Tests  at  a  changed  heater  voltage 
condition  such  as  reducedr-heater  plate  current,  trans conductance  ox 
power  output  and  high-filament  grid  current  are  made  by  first  testing 
the  tubes  at  normal  conditions  and  then  rerunning  them  in  the  same 
order  under  the  changed  condition  and  with  all  tests  except  those 
desired  skipped  out." 

The  mamual  operations  that  must  be  performed  in  conjunction  with  the 
tester  are  then  described  and  illustrated  in  a  step-by-step  manner. 

^flhe  control  section  of  this  set  contains  many  other  provisions  of 
most  importance  to  the  proper  timing  and  flow  of  information  through 
the  system....  A  manual-automatic  switch  is  provided  which,  when  in 
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the  manual  position,  deactivates  all  the  automatic  test-cycling  cir¬ 
cuits  and  permits  manual  indexing  through  the  test  cycle  by  pressing 
the  index  switch  for  each  test. 

"By  proper  arrangement  of  the  test  sequence,  data  reduction  procedure, 
and  computer  programming,  it  has  been  necessary  to  prepare  only  three 
computer  programs  in  order  to  handle  the  great  majority  of  tube  types. 

"These  include  one  program  for  the  single  triodes,  diode-trlodes, 
amplifier  pentodes  and  diode-pentodes;  and  one  for  the  dual  triodes, 
triode  pentodes,  and  duo-pentodes;  and  one  for  the  power  triodes  and 
pentodes  and  several  special  purpose  types. 

"Computations  made  by  the  computer  include  sample  averages  and  standard 
deviations  for  all  normally  distributed  characteristics;  calculation 
of  anqplification  factor,  when  required,  from  the  CM  and  HP  readings 
for  each  tube;  per  cent  deltas  with  change  of  filament  voltage  or 
over  time;  and  the  number  of  rejects  for  each  characteristic.  These 
calculation  results  are  all  printed  out  for  immediate  use. 

"Overall  accuracy  is  within  1  per  cent  for  dc  measurements  and  within 
2  per  cent  for  ac. 

"In  conclusion,  the  development  of  the  automatic  recording  test  set 
has  provided  a  means  for  accumulating  the  vast  amounts  of  test  data 
for  electron  tubes  demanded  by  current  quality  control  techniques 
in  shorter  time,  at  less  cost,  at  better  accuracy,  and  at  a  reduction 
in  the  demands  on  the  test  set  operator  whose  prime  requirement  has 
been  reduced  to  one  of  mechanical  dexterity.  Also,  it  has  provided 
the  means  for  obtaining  a  great  amount  of  additional  information  about 
tube  quality  past  and  present  at  the  cost  of  a  slight  Increase  in 
testing  and  computer  time. 

"It  is  definitely  a  major  step  toward  automation  of  the  test  function 
in  the  manufacture  of  electron  tubes  and  an  assist  in  holding  or 
reducing  testing  costs  In  spite  of  the  ever  increasing  demand  for 
more  testing  and  statistical  computations  on  the  test  data.” 

92.  Hew  York  University,  Lecture  Notes  for  the  Design  Course  in  Automatic 
Electronic  Test  Equipment,  Vols.  1  and  2,  August  29-September  2,  I960, 
352  and  205  pp. 

The  purpose  of  the  course  was  to  acquaint  designers  with  a  variety  of 
techniques  being  applied  to  checkout  systems.  Several  checkout 
arrangements  were  described  In  detail  by  the  technical  people  who  had 
originated  or  implemented  them.  Usually,  the  maintenance  philosophy 
was  ljqposed  on  the  designer;  be  was  told,  for  Instance,  that  the 
system  was  to  work  as  a  closed  loop.  Most  presentations  present 
post  hoc  arguments  justifying  the  design  choices  and  compromises  that 
were  made.  Among  the  systems  treated  are  the  following: 
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1.  Dynamic  accuracy  test  set 

2.  IFF  system  tester 

3.  BMEWS  system  checkout 

4.  Organizational  missile  test  set  (Sergeant  missile) 

5.  Terrier-Tartar  test  equipment 

6.  MD-7  fire  control  system  test  equipment  (B-58) 

7.  Flight  control  systems  test  set  (B-58) 

8.  Aircraft  and  weapon  control  system  test  set  (F-106) 

93.  Payne,  H.  H.,  ORDVAC  Reversing  Leapfrog  Tests,  Ballistic  Research 

Laboratories,  Uemo  'too.  m,  'October,  1555, '"22  BP . 

This  report  describes  a  "leapfrog"  test  for  computer  checkout. 

94.  Rocket,  F.  A.,  How  Do  You  Know  Your  Computer  Is  Working  Right?,  IBM 
Data  Systems  Division,  TN  (X).  11081.399,  November,  19597  14  pp. 

General  schemes  for  performing  tests  on  AND  and  OF  circuits  are  de¬ 
rived.  When  AJND  and  OR  gates  are  combined  in  a  network,  it  may  not 
be  possible  to  sense  intermediate  points  in  the  network.  However, 
a  rigorous  test  of  the  network  can  be  made  if  "(1)  each  AND  circuit 
is  cheeked  far  its  ability  to  prevent  an  output,  (2)  each  OR  circuit 
input  is  checked  for  its  ability  to  cause  an  output,  and  (3)  the 
system  output  is  checked  for  the  possibility  that  it  may  be  up  when 
it  should  be  down  or  down  when  it  should  be  up. " 

95.  Rocs,  D.  T.,  A  Revised  Diagnosis  Routine  for  Whirlwind  I  programs, 
Massachusetts  Institute  of  Technology,  Servo  Mschanical  Laboratories, 
Report  7138-R-l  (n.d.),  23  pp. 

A  computer  routine  that  examines  other  routines  for  mistakes  is 
described.  At  selected  points  in  the  program  under  examination  print¬ 
outs  are  aade  of  the  number  of  times  various  operations  are  performed, 
the  contents  of  various  registers,  etc.  This  allows  the  programmer 
to  troubleshoot  his  programs. 

96.  Rome  Air  Development  Center,  Final  Report  On  Study  of  Fault  Location 
and  Prediction  Techniques  Leading  to  Flans  for  Automatic  and  Dynamic 
Testing  of  Military  Ground  Equipment,  RAflc-fR  59-248  A&B,  February, 
1^66;  ■two  volumes,  156  pp.  and  195  pp. 

This  report  examines  the  design  of  "test  systems  having  Inherently  the 
capability  for  full  performance  monitoring  and  checkout  of  a  whole 
family  of  complex  systems,  and  also  for  prompt  diagnosis  of  both 
catastrophic  and  gradual  failures  whose  symptoms  may  be  revealed  in 
the  monitoring  or  checkout  process."  The  practical  development  of 
an  Automatic  Fault  Locator  (AFL)  is  discussed. 

As  to  accuracy,  an  AFL  should  have  "the  ability  to  detect  and  isolate 
significant,  catastrophic  system  faults,  and  in  addition  the  ability 
to  detect  (perhaps  very  Indirectly)  submarginal  circuit  or  unit 
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performance,  but  in  Implementing  the  latter  capability  the  accuracy 
and  precision  sought  will  be  only  that  sufficient  to  give  a  warning 
of  malfunction  to  operating  or  maintenance  personnel." 

Compromises  must  be  made  in  the  flexibility  and  degree  of  thoroughness 
of  AFL  performance,  because  ideal  flexibility  and  thoroughness  would 
require  "an  Instrument  of  enormous  complexity  and  considerable  bulk. 

The  probabilities  of  failure  within  the  AFL  system  would  at  least 
approximate  those  of  failure  within  many  of  the  equipments  which  it 
would  be  called  up  to  monitor  and  check. . .and. . .this  circumstance 
would  lead  to  confusion  and  distrust  on  the  pert  of  personnel  con¬ 
cerned  with  its  operation." 

An  AFL  should  serve  as  a  "fault  locator  or  failure  indicator,  rather 
than  as  an  instrument  which  spells  out  exactly  what  change  needs  to 
be  made  in  what  component  value  to  restore  an  equipment  to  optimum 
operational  status."  The  equipment  should  be  general  enough  to  handle 
most  electronic  signal  generation,  sensing,  and  evaluation  processes; 
in  particular,  the  "nerve  center”  or  sequence  control  should  be 
capable  of  supplementation  by  new  sensor  and  transducer  techniques 
without  requiring  redesign  of  the  master  sequencer  Itself. 

The  study  indicates  that  a  single,  general-purpose  AFL  can  cover  a 
broad  spectrum  of  fault  location  tasks,  and  that  it  can  serve  well 
either  "as  a  performance  monitor  at  the  scene  of  operations,  or  (2) 
as  a  very  useful  device  for  expediting  maintenance  and  repair  at  a 
depot."  In  view  of  the  differences  between  "on-line"  and 'bff -line" 
requirements,  however,  it  might  be  more  economical  to  pursue  two  lines 
of  AFL  development.  The  on-line  AFL  system,  for  example,  may  demand 
time- sharing  and  other  arrangements  for  sampling  parameters  without  dis¬ 
torting  on-line  performance;  these  conqplexities  would  not  be  necessary 
for  a  shop-model  AFL.  Also,  the  depth  of  fault  location  might  tend 
toward  the  module  level  for  on-line  AFL  and  the  component  or  sub-stage 
level  for  the  shop  AFL. 

The  AFL  concept  assumes  that  the  equipments  under  test  (EUT)  are 
electrical  or  electronic;  even  though  the  ultimate  parameter  being 
measured  may  be  chemical,  acoustic,  or  mechanical,  the  AFL  will 
transduce  the  non-electrical  data  into  electrical  phenomena.  Testing 
is  then  performed  on  the  "transfer  function”  of  the  separate  elements. 
The  transfer  function  is  the  mathematical  term  relating  output(s)  to 
input (s)  for  a  given  circuit,  unit,  or  system.  Since  the  proper 
operation  of  a  set  of  circuit  elements  depends  "directly  on  whether 
the  transfer  functions  are  within  acceptable  tolerances,  therefore 
fault  detection  and  isolation  is  a  process  of  determining  whether 
a  known  stimulus,  when  applied  at  a  given  point,  produces  a  particular 
prescribable  output  response  at  another  given  point."  The  transfer 
function  approach  to  test  and  evaluation  is  believed  to  reduce  the 
risk  of  obsolescence,  since  testing  is  directed  not  at  physical 
properties  (which  might  change  with  new  processes)  but  rather  at  the 
acceptability  of  a  response  produced  by  a  prescribed  excitation. 
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The  equipments  for  which  the  AFL  is  applicable  may  be  grouped  into 
four  main  categories:  (1)  coianunications,  (2)  data  processing, 
recording,  storing,  transcribing,  and  computing,  (3)  power  production, 
conversion,  and  distribution,  and  (4)  excitation,  control,  or 
monitoring  of  non-electrical  phenomena  or  processes. 

Power  equipments  are  believed  to  be  least  challenging  to  AFL,  because 
of  relatively  simple  circuitry  and  input-output  quantities.  In  order 
of  increasing  difficulty  of  AFL  development,  data  handling  equipments 
are  probably  next;  many  signal  paths  and  circuit  elements  are  usually 
encountered  and  sophisticated  sequencing  is  necessary.  On  the  other 
hand,  except  for  analog  computers,  many  of  the  parameter  states  are 
discrete,  repetitive,  and  not  frequency-dependent,  features  which 
are  favorable  to  automatic  operation.  Communication  equipments  are 
apt  to  present  a  'Variety  of  admittance  configurations  which  the 
AFL  would  have  to  evaluate.  Also,  both  the  normal  signal  inputs 
and  internal  driving  functions  in  the  Communications  group  are 
probably,  in  general,  more  complicated  time  functions  than  those  in 
any  other." 

Just  because  a  system  is  a  newly  designed,  "edge-of-the-art"  arrange¬ 
ment  does  not  guarantee  that  it  will  have  an  extremely  large  number 
of  checks.  "The  number  of  significant  inputs  and  outputs  of  an 
elaborate  surveillance  system  used  in  countermeasures  procedures 
would  likely  be  less  than  that  of  a  relatively  simple  telephonic 
switchboard  or  multlprocess  monitoring  system.”  The  sheer  number 
of  point-to-point  transfers  to  be  checked  1b  not  a  critical  AFL 
limitation.  It  is  true  that  a  large  number  of  transfers  necessitates 
a  longer  total  check  time,  "but  since  the  AFL  system  as  conceived  is 
capable  of  very  rapid  sequencing,  it  is  likely  that  the  recurrence 
rate  for  total  BUT  test  will  be  determined  more  by  the  needs  of  the 
situation  than  by  any  limitation  of  the  AFL.” 

What  if  the  principal  transfer  functions  are  variable  functions  of 
such  parameters  as  time,  frequency,  or  operator  control  action?  In 
these  cases,  the  sensing  and  excitation  are  still  likely  to  be 
straightforward,  with  the  reed.  AFL  problem  being  the  analysis  of 
the  data  from  the  transducers.  On  the  other  hand,  for  transfer 
functions  where  the  response  must  fall  within  extremely  narrow  limits, 
the  processing  is  apt  to  be  easy  and  the  measurement  process  may  be 
expensive  and  impractical.  For  tuned,  high-Q  circuits,  special 
problems  in  probe  isolation  and  close  frequency  control  may  be  en¬ 
countered. 

If  the  EUT  operates  full  time,  the  ease  of  instrumenting  an  AFL 
scheme  may  be  contingent  on  "dead  time"  in  the  duty  cycle,  for  it 
may  be  only  during  these  brief  Intervals  that  signal  insertion  and 
sampling  can  be  achieved.  It  may  be  necessary  for  the  AFL  to  sense 
and  evaluate  environmental  stimuli,  if  these  are  closely  related  to 
equipment  performance . 
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The  algebraic  requirements  for  failure  detection  are  stated:  "in  gen¬ 
eral,  one  set  of  excitation  and  response  measurements  is  sufficient  to 
determine  unambiguously  whether  a  system  parameter  is  within  or  out- 
of-tolerance  only  if  it  is  possible  to  establish  an  equation  where 
the  parameter  appears  as  a  single  unknown."  Ihe  generalization  follows: 
"a  definitive  symptom  of  failure  of  some  element  within  a  module  or 
unit  will  be  revealed,  if  at  all,  only  in  a  set  of  'n  1  relationships 
linking  excitations  and  responses,  where  'n  '  is  the  number  of  circuit 
elements— or  more  precisely,  circuit  parameters— within  the  module  or 
unit. " 

This  "n-variable"  logic  can  be  extended  to  the  actual  circuit  case, 
and  there  is  a  possibility  of  exciting  a  module  with  enough  signals 
to  solve  for  the  values  of  the  different  components  in  the  module. 
Logically,  then,  piece-part  fault  identification  within  the  module  can 
be  accomplished  positively  by  suitable  computer  programming. 

Despite  the  basic  appeal  of  such  computer-controlled  schemes,  various 
difficulties  are  evident.  Ihe  procedures  do  not  work  well  with  non¬ 
linear  elements;  usually  the  assumption  of  linearity  must  be  made 
for  both  the  normal  circuit  and  the  fault  condition.  Physical  isolation 
of  the  module  Itself  may  be  necessary,  if  the  input-output  transfer  is 
to  be  critically  examined.  Ihe  number  of  unknowns  in  the  module  may 
far  exceed  the  number  of  parts.  "Some  significant  percentage  of  faults 
in  any  system  is  traceable  not  to  departure  from  required  value  of  a 
given  capacitance,  resistance,  or  inductance,  but  to  the  fact  that 
(because  of)  fatigue,  vibration,  environmental  conditions,  etc., 
significant  impedances  and/or  admittances  cone  to  exist  in  the  paths 
that  Interconnect  circuit  elements . " 

For  simple  systems,  it  is  usually  apparent  to  the  operator  when  fail¬ 
ure  occurs .  With  more  coajplex  systems,  the  operator  may  have  no  way 
of  knowing  whether  the  system  is  performing  adequately,  if  certain 
superficial  aspects  remain  stable.  This  calls  for  a  fairly  continuous 
over-all  system  check.  Thus,  "it  would  be  highly  desirable  in  an 
automated  radar  test  system  to  be  able,  by  air  link  and  microwave  hour, 
to  Introduce  into  the  antenna  system  a  mint ml -level  simulated  target 
and  automatically  to  detect  whether  an  appropriately  coded  response 
appeared  at  the  output  of  the  target— data  digitizer. "  Such  over-all 
checking  methods  are  also  naturally  adaptable  to  AIL  self-checking, 
and  they  will  tend  to  reduce  the  necessity  dor  detailed  checking  of 
operator-varied  controls. 

Several  operational  constraints  for  the  AFL  concept  are  mentioned. 

In  the  first  place,  the  BUT  should  not  be  disturbed  in  such  a  way 
as  to  impair  system  performance;  this  iaplies  reliable  interlock 
provisions  and  tight  coupling  between  EOT  duty  cycle  and  the  AFL  test 
regime.  A  second  ground  rule  is  the  "avoidance  of  methods  requiring 
analytical  judgment  and  physical  manipulation  by  operating  personnel," 
presumably  in  the  interest  of  standardization  and  efficiency.  Third, 
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tests  requiring  the  EOT  to  be  de-energized  should  be  avoided,  (it  is 
recognized  that  some  such  checks  may  be  Indispensable.) 

It  is  desirable  for  the  sensed  test  data  from  all  test  points  to  be 
in  the  same  analog  form.  Of  the  many  possible  ways  of  expressing  the 
analogs,  unipolar  voltages  axe  considered  to  be  the  most  advantageous. 
Voltages  are  manageable  for  machine  comparisons;  they  can  be  picked 
up  over  low  impedances,  they  can  be  expected  to  be  relatively  indepen¬ 
dent  of  EOT  duty  cycle,  and  they  can  be  easily  transmitted  from  dis¬ 
persed  locations  to  a  central  API  comparator. 

Given  a  uni  polar  voltage  analog  as  the  typical  analog  test  parameter, 
four  main  classes  of  failure  tests  can  be  distinguished:  Class  A, 
power  inputs  and  outputs;  Class  B,  checking  of  Internally  generated 
control  voltages,  triggers,  gating  step-functions,  etc.;  Class  C,  in¬ 
put  simulation  methods;  and  Class  D,  environmental  testing. 

Each  class  of  test  has  its  own  problems  of  AFL  implementation.  Class 
A  tests  will  probably  be  the  easiest  to  implement,  and  can  be  expected 
to  provide  sufficient  diagnostic  information  for  about  half  of  the 
failures .  Class  B  tests,  even  though  they  have  no  separate  excitation, 
may  require  complex  conversion  circuitry.  Class  C  tests  will  probably 
be  the  most  difficult  to  implement,  because  of  the  precise  stimulation 
signals  that  must  be  provided.  Some  trade-offs  between  accuracy  and 
coverage  can  be  made;  a  scheme  that  tests  only  60  per  cent  of  the 
significant  transfer  functions  but  does  these  with  high  accuracy  may 
be  preferable  to  a  100  per  cent  coverage  and  a  higher  error  rate. 
Generalizations  about  Class  D  or  environmental  checks  are  difficult 
because  of  the  range  of  situational  variables.  Equipment  performance 
degradation  generally  lags  environmental  conditions;  therefore,  "Class 
D  tests  have  fault  prediction  capabilities;  an  indication  of  out-of¬ 
tolerance  of  system-generated  environmental  conditions  is  a  signal  to 
take  corrective  action." 

A  failure  might  be  described  by  tests  of  more  than  one  class.  An 
illustration  case  is  a  failure  in  a  gated  stage  of  a  video  amplifier . 

"A  Class  B  test  might  reveal  this  fed  lure  in  an  out-of-tolerance  con¬ 
dition  with  respect  to  the  gating  voltage  applied  to  the  gating  grid 
in  the  stage.  Likewise,  a  Class  C  test  might  reveal  the  failure  in 
terms  of  an  out-of-tolerance  condition  of  the  primary  signal  transfer 
function  linking  video  input  to  video  output." 

Processing  of  the  results  from  these  different  classes  of  test  would 
generally  proceed  in  a  hierarchical  manner.  Thus,  if  a  failure  caused 
out-of-tolerance  findings  for  many  test  points,  the  machine  analysis 
sequence  is  to  investigate  Class  A  results  first,  and  not  to  proceed 
to  B  and  C  unless  required,  in  the  case  where  a  Class  A  fault  is 
cleared  but  Class  B  and  Class  C  out— of- tolerance  indications  persist, 
then  the  conclusion  is  that  mare  than  one  system  transfer  function  was 
out-of-tolerance . 
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Empirical  determination  of  tolerance  limits  seems  to  be  inescapable  if 
the  system  to  be  automated  is  at  all  complicated.  Ibis  is  because  the 
transfer  functions  may  have  "non-linearities  and  inexactness  which  in¬ 
herently  make  it  very  hard  to  state  with  assurance  in  advance  just 
what  values  of  the  final  form  of  the  converted  data  correspond  to  in¬ 
tolerance  conditions."  Provision  should  be  made  for  competent  install¬ 
ation  personnel  to  make  on-site  adjustments  of  tolerances. 

Test  timing  will  not  be  difficult  for  Class  A,  Class  D,  and  some  Class 
B  tests.  For  many  Class  B  and  C  tests,  however,  special  saopllng  and 
processing  provisions  must  be  made.  Perhaps  the  worst  case  to  be  en¬ 
countered  is  when  the  signal  Inputs  are  random  in  time  and  the  BUT 
cannot  be  shut  down  for  Class  C  tests.  Ibis  condition  implies  sensing 
circuits  that  recognize  whether  a  signal  is  present,  as  well  as  a 
programmed  sequence  for  running  tests  while  no  "real"  signals  are  there. 
Ideally,  this  test  sequence  could  be  interrupted  by  a  sensed  signal  and 
still  resume  the  tests  later  without  losing  any  of  the  earlier  test 
information.  If  a  simple  'bolding  circuit"  were  available  for  holding 
voltage  levels  over  low  ispendances,  then  "integrating  Class  B  and  C 
tests  into  the  overall  test  program  would  be  nearly  as  easily  accomplished 
as  Integrating  Class  A  and  Class  D  tests." 

Two  basic  types  of  fault-isolating  sequences  cure  distinguished.  Ibe 
branching  method  is  a  successive-contingency  form  of  analysis;  the 
steps  have  been  carefully  programmed  in  advance.  Ihus,  the  programed 
series  is  altered  according  to  the  test  data  obtained,  and  "doses  in" 
on  the  trouble.  Branching  may  extend  to  the  initiation  of  new  checks 
that  were  not  in  the  basic  program.  The  so-called  simple  ordering 
method  is  more  of  a  coincidence  technique.  No-go  readings  at  the 
different  test  levels  (Class  A,  B,  C,  and  D)  are  studied,  and  a  pattern 
of  these  discrepent  readings  related  to  a  hypothesized  trouble.  Pre¬ 
liminary  trials  suggest  that  the  ordering  method  may  be  more  effective 
than  a  random  component  model  would  indicate. 

Eight  procedures  can  be  followed  in  designing  new  equipment  to  make 
the  system  amenable  to  AFL  Instrumentation: 

1.  Use  modularization  whenever  possible 

2.  Avoid  dual  functions  in  the  same  module 

3.  Provide  space  for  miniature  local  transducers  and  exciters 

4.  Incorporate  high-to-low  impedance  conversion  devices 

5 •  Implement  electronic  or  electrical  means  for  establishing 
value  of  system  parameters 

6.  Whenever  warning  lights,  meters,  and  other  indicators  are 
to  be  designed  into  the  system,  provide  a  voltage  analog 

7.  Require  original  equipment  designers  to  be  aware  of  the 
excitation  and  sensory  necessities  for  Class  C  tests 

8*  Devise  marginal  check  parameters 
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97.  Scully,  J.  F.,  and  L.  P.  Colangelo,  "Field  Performance  of  a  New  Auto¬ 
matic  Fault-Locating  Means,"  proceedings  of  the  Western  Joint  Com¬ 
puter  Conference,  February,  l95?>  PP*  211-214. 

"An  automatic  fault-locating  means  has  been  devised,  reduced  to  practice, 
and  subsequently  utilized  in  a  large-scale  digital  da  tap-processing 
equipment  built  for  the  Rome  Air  Development  Center  by  the  Monroe 
Calculating  Machine  Company.  The  machine  solves  a  classified  Air 
Force  problem  and  contains  several  thousand  logical  elements  (vacuum 
tubes  and  diodes) .  The  Monrobot  Automatic  Internal  Diagnosis  (MAID) 
monitors  the  machine  at  all  times  to  make  sure  that  there  are  no  cir¬ 
cuit  failures  which  might  cause  errors  in  the  solution  of  the  problem. 
Upon  occurrence  of  a  failure,  the  diagnosis  unit  quickly  and  automat¬ 
ically  localizes  the  circuit  at  fault. 

"The  genesis  of  this  automatic  fault-location  system  lay  in  the  answer 
to  the  question:  since  electronic  calculators  have  been  designed  to 
perform  automatically  operations  otherwise  handled  by  humans,  why  do 
we  not  design  maintenance  machines  to  replace  servicemen?  MAID  is  a 
pioneer  answer  to  this  question." 

As  to  the  fault-locating  logic,  "Self  repair  implies  that  maintenance 
operations  are  completely  mechanized.  Five  steps  are  required  where 
maintenance  is  to  be  done  on  electronic  calculating  or  switching  cir¬ 
cuits:  error  detection,  fault-location,  component  replacement,  error 
clearing,  and  restarting.  With  the  exception  of  component  replacement, 
each  step  is  easily  susceptible  of  mechanization.  The  system  here 
considered  mechanizes  these  steps  and  so  greatly  reduces  the  time  re¬ 
quired  to  repair  a  fault  when  it  occurs .  It  does  so  by  scanning 
points  of  possible  error  in  the  machine.  If  one  point  exhibits  de¬ 
fective  behavior  earlier  than  another,  it  is  regarded  as  a  "better" 
cause  of  the  error  than  the  other;  if,  on  the  other  band,  two  points 
exhibit  the  error  simultaneously,  that  point  which  is  functionally 
independent  of  the  other  is  taken  as  the  "better"  cause.  When  all 
points  of  the  machine  have  been  scanned,  one  point  stands  out  as  the 
cause  of  all  of  the  errors  which  were  observed  during  the  scanning 
process.  Phrased  alternatively,  we  may  say  that  of  all  points  of 
possible  error,  sane  subset  of  points  will  exhibit  the  error  at  the 
earliest  time;  of  this  subset,  that  point  which  is  functional ly  least 
dependent  on  the  others . locates  the  cause.  A  decimal  number  assigned 
to  this  point  appears  in  lights  on  the  control  console;  replacement 
of  the  plug-in  unit  associated  with  this  number  by  a  spare  completes 
the  repair  and  permits  correct  operation  to  continue." 

Operating  experience  with  the  MAID  Indicates  that  substantial  gains 
were  realized  over  normal  troubleshooting  techniques,  and  times  were 
approximately  halved. 

Other  benefits  are  hypothesized:  "The  automatic  diagnosis  means  em¬ 
ployed  has  additional  advantages  not  reflected  directly  in  the 
efficiency  figures.  One  of  the  most  important  of  these  is  the 
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reduction  in  the  level  of  training  required  to  maintain  the  equipment. 
Since  the  difficult  task  of  locating  troubles  is  now  mechanized,  most 
troubles  can  be  serviced  by  personnel  with  very  little  technical  skill 
who  could  not,  without  this  aid,  service  the  equipment  at  all.  Second¬ 
ly,  as  a  production  trouble-shooting  means,  impressive  reductions  in 
delivery  schedules  can  be  made  with  no  increase  in  technical  staff,  a 
matter  of  great  importance  in  these  times  of  critical  shortage.  Thirdly, 
training  of  personnel  to  a  high  degree  of  efficiency  is  made  much  easier, 
as  the  machine  itself  does  much  of  the  teaching!  Also  of  great  im¬ 
portance  is  the  security  inherent  in  the  certain  knowledge  that  an 
equipment  is  performing  correctly  at  a  given  time. 

"The  application  of  automatic  diagnosis  discussed  here  is  the  first  in 
the  art.  We  believe  that  it  has  fully  lived  up  to  expectations,  but 
recognize  at  the  same  time  that  improvement  is  certainly  possible.  In 
the  first  place,  the  desirability  of  extension  of  the  principle  to  in¬ 
clude  complete  systems,  rather  than  portions  only,  was  clear  from  the 
outset.  The  practical  limitations  which  weighed  against  doing  this  in 
the  prototype  have  since  been  removed  and  succeeding  calculators  have 
been  built  with  the  completely  automatic  system.  The  advantages  of 
doing  this  have  been  realized  in  practice.  Also,  it  has  been  recognized 
that  MAID  application  should  be  considered  as  a  vital  part  of  the  de¬ 
sign  of  equipment  if  its  full  potentialities  are  to  be  achieved. 
Experience  has  shown  that  applique  construction  is  often  possible,  but 
not  as  efficient.  Again,  since  the  past  experience  of  the  people  con¬ 
cerned  has  been  largely  in  the  field  of  digital  techniques,  it  was 
natural  to  devote  the  major  application  effort  in  this  field;  however, 
it  must  be  stressed  that  the  means  are  applicable  to  many  nondigital 
electronic  systems.  Work  is  progressing  to  Improve  the  actual  means 
employed  so  that  both  simpler  circuitry  and  even  more  rapid  fault 
analysis  can  be  made.  Consideration  has  also  been  given  to  more  direct 
correlation  of  trouble  indication  with  defective  component  to  still 
further  simplify  the  duties  of  maintenance  personnel." 

98.  Sell,  R.  P.,  "Program  Debugging  for  parallel  Processing,"  paper  pre¬ 
sented  at  the  National  Conference  of  the  Association  of  Computing 
Machinery,  August  25-26,  i960. 

Parallel  processing  is  described:  "The  Honeywell  800  provides  for 
automatic  parallel  processing.  The  term  'automatic'  is  used  here  be¬ 
cause  up  to  8  programs  may  be  run  on  the  system,  without  the  use  of 
any  monitoring  program.  The  sequencing  of  these  programs  is  automatic 
through  Inclusion  of  a  control  memory  and.  special  built-in  equipment 
called  Mil ti-Program  Control  and  Traffic  Control." 

In  the  design  of  this  debugging  system  for  the  Honeywell  800,  "two 
criteria  were  of  overriding  importance: 

1.  The  checkout  system  should  be  capable  of  processing  any 
number  of  programs  in  a  checkout  run  with  a  minimum  of 
human  Intervention  and  thus  with  a  minimum  of  idle  machine 
time. 
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2.  The  checkout  system  should  not  require  any  changes  to 
a  program  solely  for  the  purpose  of  debugging.  At 
the  same  time,  the  system  should  furnish  the  programmer 
with  enough  diagnostic  Information  so  that  a  nartimim 
of  errors  could  be  corrected  after  given  checkout  run. 

"These  basic  techniques  in  the  Honeywell  800  checkout  system  are  appli¬ 
cable  to  serial  computers  Just  as  well  as  to  a  multi  -program  computer. 
The  latter,  however,  makes  highly  desirable  additional  techniques 
which  take  account  of  parallel  processing.  While  self-destructive 
programs  have  been  with  us  since  the  beginning  of  the  computer  age, 
parallel  processing  introduced  the  possibility,  remote  as  it  may  be, 
of  mutually  destructive  programs.  Interference  between  programs  in 
parallel  processing  may  take  one  of  three  forms: 

(1)  Control  branches --as  far  as  the  originating  program 
is  concerned— into  'wilderness.'  If  this  'wilderness' 
is  the  assigned  memory  area  of  a  parallel-processed 
program,  unpredictable  results  may  occur. 

(2)  A  program  stores  or  reads  information  into  memory  areas 
occupied  by  a  parallel-processed  program.  This 
'clobbering'  of  one  or  more  words  in  another 'program 
may  not  disburb  the  operation  of  the  originating 
program  but  it  could,  probably  would,  seriously  inter¬ 
fere  with  the  proper  operation  of  the  offended  program. 

(3)  A  program  refers  destructively  to  one  or  more  unique 
registers  in  the  centred  processor  like  Accumulator, 

Low  Order  Product  Register,  Mask  Register,  or  Stopper 
Address.  If  the  contents  of  such  a  register  were  re¬ 
quired  in  subsequent  operations  by  a  parallel-processed 
program  incorrect  results  could  be  developed." 

99.  Stone,  K.  0.,  P.  Meissner,  and  K.  M.  Schwarz,  The  Expandability  of 
Electronic  Asseablies,  National  Bureau  of  Standards,  UBS  Report 
ol«3  (n.d. ) 

"The  optimum  module  which  would  lead  to  lowest  overall  procurement 
cost  and  at  the  same  time  be  of  aid  in  the  solution  of  logistic  and 
maintenance  problems  would  be  one  containing  from  4  to  8  tubes. 

This  is  true  for  both  disposal -at -failure  and  repairable  modules." 

100.  lyndall  Air  Force  Base,  Evaluation  of  the  UG  897  Automatic  Test  Set, 
Project  ADC/73AD/59-12,  October,  1959. 

An  evaluation  of  the  UG  897  was  conducted  by  using  the  set  to  find 
inserted  and  actual  malfunctions  in  the  F-101B  Automatic  Flight 
Control  System  (AFCS)  and  Central  Air  Data  Conputer  (CADC). 

"The  UG  897B-I  proved  both  highly  accurate  and  reliable  to  an  ex¬ 
treme  not  often  found  in  this  type  of  equipment.  During  the  evalu¬ 
ation,  the  test  set  accurately  located  all  induced  and  actual  flight 
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malfunctions  assigned  to  it.  Use  of  the  set  reduced  average  flight 
line  tremble-shoot  time  by  an  estimated  8o£  and,  in  aany  cases, 
eliminated  a  functional  check  flight.  This  represented  a  consider* 
able  savings  in  manpower,  flight  time  and  dollars.  The  tester  haa 
considerable  growth  potential  toward  cheeking  other  electrical  sys¬ 
tems  and  aircraft. 

"The  X  897B-I  can  replace  all  existing  F-101B  AFCS  flight  line  test 
equipment  and  do  the  job  faster  with  greater  dependability.  With  the 
aid  of  a  suitable  pneumatic  pressure  source,  the  same  statement  may 
be  made  of  its  ability  to  check  out  the  CADC.  In  its  present  con* 
figuration,  however,  it  can  test  only  the  electrical  circuitry  with¬ 
in  the  CADC,  covering  only  an  estimated  80  to  90  percent  of  the 
usual  'flight  squawks'." 

101.  Uchlyamada,  H. ,  Comprehensive  System  of  Service  Routines.  Digital 
Computer  Laboratories,  Massachusetts  Institute  of  Technology, 

December  17,  1952  (revised  1953),  31  PP- 

The  word  "service"  refers  to  the  editing,  program  change,  and  input/ 
output  functions.  No  reference  is  made  to  checkout  or  maintenance. 

102.  Williams,  C.  R.,  A  Review  of  ORDVAC  Operating  Experience .  Ballistic 
Research  laboratories,  Memo  No.  756,  January,  1994,19  pp. 

Three  kinds  of  checkout  tests  are  described: 

1.  A  "read-areund  test"  to  check  memory  (this  is  a 
requirement  peculiar  to  electrostatic  memories  where 
repeated  reference  to  one  memory  location  may  in¬ 
fluence  adjacent  locations) 

2.  Input/ output  tests 

3.  "Leap  III,"  an  arithmetic,  control  and  memory  test 

103.  Wcblg,  W.  H. ,  and  L.  E.  Tryon,  Fault  Location  Study.  RADC-TR-59-250, 
December,  1959,  2  Vola.,  149  pp.  plus  2  appendixes  and  90  pp. 

This  investigation  concerned  the  feasibility  of  automatic  fault 
location  equipment  for  two  existing  SACK  radars  (Search  RADAR  FP6-20 
and  Height  Finder  Radar  FPS-6),  and  parallels  item  96. 

Only  two  fault  isolation  concepts  are  considered.  One  is  the  "simple 
ordering"  idea:  go/no-go  results  are  read  continuously  from  the  whole 
collection  of  test  points;  to  analyze  these  indications,  a  fixed 
routine  is  programmed  and  interpretation  of  the  data  takes  relatively 
little  advantage  of  redundancies  or  serial  dependencies  in  the  test 
scheme.  This  simple  ordering  method  is  rejected  in  favor  of  logical 
sequencing  or  branching.  In  this  latter  scheme,  the  decisions  to 
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make  another  test  and  vhich  test  to  make  are  contingent  upon  the 
results  of  preceding  testu.  Thus  the  program  "pursues"  the  trouble 
instead  of  trying  to  find  it  by  fixed  comparisons.  In  general,  signal 
voltages  only  are  sampled,  since  supply  voltages  can  be  monitored  by 
communtator-comparator  methods  at  the  power  sources.  No-go  indications 
from  the  commutators  could  then  cause  the  main  fault  location  program 
to  be  inhibited  until  the  supply  voltages  were  at  satisfactory  levels. 

Most  branching  tests  would  be  either  go/no-go  or  hi/lo/go.  A  few 
intrinsically  difficult  tests  were  noted,  (e.g.,  velocity  response 
shaping  characteristics),  and  problems  in  setting  limits  were  re¬ 
garded  as  "difficult  but  amenable."  Open- loop  tests  on  servomechanisms 
are  regarded  as  undesirable  since  performance  interruption  would 
usually  occur  and  a  decision  must  be  made  by  operating  personnel  as  to 
whether  or  not  operation  should  be  interrupted  to  permit  further 
fault  isolation  procedures  on  an  open- loop  basis. 

It  would  be  possible  to  have  a  central  or  master  fault  locator  that 
could  control  the  subsystem  programs  from  a  single  storage  position. 
Such  an  arrangement  would  save  individual  comparator  programmer  units. 
But  it  would  be  accompanied  by  so  many  disadvantages  (complexity, 
storage,  maintenance,  and  programming  requirements)  that  the  present 
recommendation  is  for  individual  fault  locators  that  can  later  be 
converted  to  a  central  control  arrangement. 

The  relative  merits  of  parallel  and  serial  comparators  are  investi¬ 
gated;  a  direct  qualitative  parallel  is  recommended.  Implementation 
of  the  access  speed  and  volume  of  program  storage  requirements  are 
probably  within  present  technological  capabilities. 
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